ARISTA %2D image search to annotation on billions of web photos.

Empower People with Knowledge The Next Frontier for Web Search.

Mining adjacent markets from a large%2Dscale ads video collection for image advertising.



GeoLife A Collaborative Social Networking Service among User
People travel in the real world and leave their location history in a form of trajectories. These trajectories do not only connect locations in the physical world but also bridge the gap between people and locations. This paper introduces a social networking service, called GeoLife, which aims to understand trajectories, locations and users, and mine the correlation between users and locations in terms of usergenerated GPS trajectories. GeoLife offers three key applications scenarios: 1) sharing life experiences based on GPS trajectories; 2) generic travel recommendations, e.g., the top interesting locations, travel sequences among locations and travel experts in a given region; and 3) personalized friend and location recommendation. 1
Understanding transportation modes based on GPS data for web applications.

A large%2Dscale study on map search logs.


Mining city landmarks from blogs by graph modeling.
Recent years have witnessed great prosperity in communitycontributed multimedia. Discovering and summarizing knowledge from these data enables us to make better sense of the world. In this paper, we report our work on mining famous city landmarks from blogs for personalized tourist suggestions. Our main contribution is a graph modeling framework to discover city landmarks by mining blog photo correlations with community supervision. This modeling fuses context, content, and community information in a style that simulates static (PageRank) and dynamic (HITS) ranking models to highlight representative data from the consensus of blog users. Preliminary, we identify geographical locations of page contents to harvest city sight photos from Web blogs, based on which we structure these photos into a Scene-View hierarchy * within each city. Our graph modeling consists of two phases: First, within a

Vocabulary hierarchy optimization for effective and transferable retrieval.
Scalable image retrieval systems usually involve hierarchical quantization of local image descriptors, which produces a visual vocabulary for inverted indexing of images. Although hierarchical quantization has the merit of retrieval efficiency, the resulting visual vocabulary representation usually faces two crucial problems: (1) hierarchical quantization errors and biases in the generation of “visual words&#034;; (2) the model cannot adapt to database variance. In this paper, we describe an unsupervised optimization strategy in generating the hierarchy structure of visual vocabulary, which produces a more effective and adaptive retrieval model for large-scale search. We adopt a novel Density-based Metric Learning (DML) algorithm, which corrects word quantization bias without supervision in hierarchy optimization, based on which we present a hierarchical rejection chain for efficient online search based on the vocabulary hierarchy. We also discovered that by hierarchy optimization, efficient and effective transfer of a retrieval model across different databases is feasible. We deployed a large-scale image retrieval system using a vocabulary tree model to validate our advances. Experiments on UKBench and street-side urban scene databases demonstrated the effectiveness of our hierarchy optimization approach in comparison with state-of-the-art methods. 1.
Mining correlation between locations using human location history.

Advertising based on users%27 photos.
In this paper, we tackle the problem of learning a user’s interest from his photo collections and suggesting relevant ads. We address two key challenges in this work: 1) understanding a user’s photos to detect his interest, and 2) bridging the lexical and semantic gap between the vocabulary of ads and that of general users ’ photos. We solve the first problem by employing a data-driven image annotation approach to annotate each photo and modeling a group of photos, and tackle the second problem by learning and matching the topics of users ’ photos and ads. The experiments based on real Flickr data showed the effectiveness of the approach. Index Terms — image advertising, user interest modeling 1.

Incorporating site%2Dlevel knowledge for incremental crawling of web forums a list%2Dwise strategy.
We study in this paper the problem of incremental crawling of web forums, which is a very fundamental yet challenging step in many web applications. Traditional approaches mainly focus on scheduling the revisiting strategy of each individual page. However, simply assigning different weights for different individual pages is usually inefficient in crawling forum sites because of the different characteristics between forum sites and general websites. Instead of treating each individual page independently, we propose a list-wise strategy by taking into account the site-level knowledge. Such sitelevel knowledge is mined through reconstructing the linking structure, called sitemap, for a given forum site. With the sitemap, posts from the same thread but distributed on various pages can be concatenated according to their timestamps. After that, for each thread, we employ a regression model to predict the time when the next post arrives. Based on this model, we develop an efficient crawler which is 260% faster than some state-of-the-art methods in terms of fetching new generated content; and meanwhile our crawler also ensure a high coverage ratio. Experimental results show promising performance of Coverage, Bandwidth utilization, and Timeliness of our crawler on 18 various forums.

GeoLife2.0 A Location%2DBased Social Networking Service.

Incorporating site%2Dlevel knowledge to extract structured data from web forums.
Categories and Subject Descriptors Web forums have become an important data resource for many web applications, but extracting structured data from unstructured web forum pages is still a challenging task due to both complex page layout designs and unrestricted user created posts. In this paper, we study the problem of structured data extraction from various web forum sites. Our target is to find a solution as general as possible to extract structured data, such as post title, post author, post time, and post content from any forum site. In contrast to most existing information extraction methods, which only leverage the knowledge inside an individual page, we incorporate both page-level and site-level knowledge and employ Markov logic networks (MLNs) to effectively integrate all useful evidence by learning their importance automatically. Site-level knowledge includes (1) the linkages among different object pages, such as list pages and post pages, and (2) the interrelationships of pages belonging to the same object. The experimental results on 20 forums show a very encouraging information extraction performance, and demonstrate the ability of the proposed approach on various forums. We also show that the performance is limited if only page-level knowledge is used, while when incorporating the site-level knowledge both precision and recall can be significantly improved.
Mining interesting locations and travel sequences from GPS trajectories.

Browsing on small displays by transforming Web pages into hierarchically structured subpages.

Proceedings of the 17th International Conference on World Wide Web

Information Retrieval Technology
Retrieving information from the Web is becoming a common practice for internet users. However,
Delivering online advertisements inside images.
We present in this paper a new channel to deliver online advertisements along with Web images and show a new business model to monetize billions of Web images. The idea is intuitively inspired by image displaying processes on the Web, which typically require people to wait a few seconds before they see full resolution images. This is due to large file sizes and limited network bandwidth. To utilize idle time and the display area, we propose an innovative method for non-intrusively embedding ads into images in a visually pleasant manner. To maintain a smooth user experience, we utilize the thumbnail of the full-resolution image because it is small and visually similar to the full-resolution image. At the client side, a rendering engine first enlarges and blurs the thumbnail, and then blends the pre-chosen ads information into the enlarged image. Based on this idea, we propose three typical scenarios that can adopt the proposed imageadvertising mode. More importantly, we can encourage providers of images or other users to participate in our online image ads service by tagging or annotating images. We envision revenue sharing with the providers participating in our service, and we expect that a large number of users will actively submit, tag and annotate images using the system. We have implemented a prototype image ads system, and conducted a series of experiments and user studies to evaluate such a new advertisement channel. The experimental results and user studies show that the proposed online image ad delivery is a non-intrusive ads mode, and the proposed solution is practical. This work also opens multiple new research directions ranging from multimedia to web data mining.
Flickr distance.
Abstract. In this paper, we describe the participation of TELECOM ParisTech in the ImageCLEF 2010 Photo Annotation challenge. This edition focuses on promoting combination between visual and tag features in order to enhance photo annotation. An image collection is supplied with tags which are used both for training and testing. Our training approach consists of building SVM classifiers and kernels which take into account the similarity between visual features as well as tags. The results clearly corroborate (i) the complementarity of tags and visual descriptors and (ii) the effectiveness of SVM classifiers in photo annotation. 1
Building Web%2DScale Data Mining Infrastructure for Search.

Search%2Dbased query suggestion.
In this paper, we proposed a unified strategy to combine query log and search results for query suggestion. In this way, we leverage both the users ’ search intentions for popular queries and the power of search engines for unpopular queries. The suggested queries are also ranked according to their relevance and qualities; and each suggestion is described with a rich snippet including a photo and related description.
Search%2Dbased query suggestion.
In this paper, we proposed a unified strategy to combine query log and search results for query suggestion. In this way, we leverage both the users ’ search intentions for popular queries and the power of search engines for unpopular queries. The suggested queries are also ranked according to their relevance and qualities; and each suggestion is described with a rich snippet including a photo and related description.
Density based co%2Dlocation pattern discovery.
Co-location pattern discovery is to find classes of spatial objects that are frequently located together. For example, if two categories of businesses often locate together, they might be identified as a co-location pattern; if several biologic species frequently live in nearby places, they might be a co-location pattern. Most existing co-location pattern discovery methods are generate-and-test methods, that is, generate candidates, and test each candidate to determine whether it is a co-location pattern. In the test step, we identify instances of a candidate to obtain its prevalence. In general, instance identification is very costly. In order to reduce the computational cost of identifying instances, we propose a density based approach. We divide objects into partitions and identifying instances in dense partitions first. A dynamic upper bound of the prevalence for a candidate is maintained. If the current upper bound becomes less than a threshold, we stop identifying its instances in the remaining partitions. We prove that our approach is complete and correct in finding co-location patterns. Experimental results on real data sets show that our method outperforms a traditional approach.
Mining user similarity based on location history.

Spatial pyramid mining for logo detection in natural scenes.
This work introduces a novel data mining scheme, spatial pyramid mining, to discover association rules at multiple resolutions in order to identify frequent spatial configurations of local features that correspond to classes of logos appearing in real world scenes. By indexing representative examples by the mined rules we can efficiently detect a variety of different lettering or design marks associated with a brand. Features in an image are marked by matching rules to representative examples selected via a weighted cosine similarity measure. Logos are localized in an image via density-based clustering of matched features. Precision vs. recall curves are presented for experiments on a dataset of web images of nearly 1,000 images containing seven popular logo types.
Visual pattern weighting for near%2Dduplicate image retrieval.

Automatic video annotation through search and mining.
Conventional approaches to video annotation predominantly focus on supervised identification of a limited set of concepts, while unsupervised annotation with infinite vocabulary remains unexplored. This work aims to exploit the overlap in content of news video to automatically annotate by mining similar videos that reinforce, filter, and improve the original annotations. The algorithm employs a two-step process of search followed by mining. Given a query video consisting of visual content and speech-recognized transcripts, similar videos are first ranked in a multimodal search. Then, the transcripts associated with these similar videos are mined to extract keywords for the query. We conducted extensive experiments over the TRECVID 2005 corpus and showed the superiority of the proposed approach to using only the mining process on the original video for annotation. This work represents the first attempt at unsupervised automatic video annotation leveraging overlapping video content. Index Terms — video annotation, video search, data mining 1.
Vocabulary tree incremental indexing for scalable location recognition.

A Flexible Spatio%2DTemporal Indexing Scheme for Large%2DScale GPS Track Retrieval.

GeoLife Managing and Understanding Your Past Life over Maps.

Directly optimizing evaluation measures in learning to rank.
One of the central issues in learning to rank for information retrieval is to develop algorithms that construct ranking models by directly optimizing evaluation measures used in information retrieval such as Mean Average Precision (MAP) and Normalized Discounted Cumulative Gain (NDCG). Several such algorithms including SVM map and AdaRank have been proposed and their effectiveness has been verified. However, the relationships between the algorithms are not clear, and furthermore no comparisons have been conducted between them. In this paper, we conduct a study on the approach of directly optimizing evaluation measures in learning to rank for Information Retrieval (IR). We focus on the methods that minimize loss functions upper bounding the basic loss function defined on the IR measures. We first provide a general framework for the study and analyze the existing algorithms of SVM map and AdaRank within the framework. The framework is based on upper bound analysis and two types of upper bounds are discussed. Moreover, we show that we can derive new algorithms on the basis of this analysis and create one example algorithm called PermuRank. We have also conducted comparisons between SVM map, AdaRank, PermuRank, and conventional methods of Ranking SVM and Rank-Boost, using benchmark datasets. Experimental results show that the methods based on direct optimization of evaluation measures can always outperform conventional methods of Ranking SVM and RankBoost. However, no significant difference exists among the performances of the direct optimization methods themselves.
Exploring traversal strategy for web forum crawling.
In this paper, we study the problem of Web forum crawling. Web forum has now become an important data source of many Web applications; while forum crawling is still a challenging task due to complex in-site link structures and login controls of most forum sites. Without carefully selecting the traversal path, a generic crawler usually downloads many duplicate and invalid pages from forums, and thus wastes both the precious bandwidth and the limited storage space. To crawl forum data more effectively and efficiently, in this paper, we propose an automatic approach to exploring an appropriate traversal strategy to direct the crawling of a given target forum. In detail, the traversal strategy consists of the identification of the skeleton links and the detection of the page-flipping links. The skeleton links instruct the crawler to only crawl valuable pages and meanwhile avoid duplicate and uninformative ones; and the page-flipping links tell the crawler how to completely download a long discussion thread which is usually shown in multiple pages in Web forums. The extensive experimental results on several forums show encouraging performance of our approach. Following the discovered traversal strategy, our forum crawler can archive more informative pages in comparison with previous related work and a commercial generic crawler.
Understanding mobility based on GPS data.

Rich media and web 2.0.

Annotating Images by Mining Image Search Results.
Abstract—Although it has been studied for years by the computer vision and machine learning communities, image annotation is still far from practical. In this paper, we propose a novel attempt at model-free image annotation, which is a data-driven approach that annotates images by mining their search results. Some 2.4 million images with their surrounding text are collected from a few photo forums to support this approach. The entire process is formulated in a divide-and-conquer framework where a query keyword is provided along with the uncaptioned image to improve both the effectiveness and efficiency. This is helpful when the collected data set is not dense everywhere. In this sense, our approach contains three steps: 1) the search process to discover visually and semantically similar search results, 2) the mining process to identify salient terms from textual descriptions of the search results, and 3) the annotation rejection process to filter out noisy terms yielded by Step 2. To ensure real-time annotation, two key techniques are leveraged—one is to map the high-dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Since no training data set is required, our approach enables annotating with unlimited vocabulary and is highly scalable and robust to outliers. Experimental results on both real Web images and a benchmark image data set show the effectiveness and efficiency of the proposed algorithm. It is also worth noting that, although the entire approach is illustrated within the divide-andconquer framework, a query keyword is not crucial to our current implementation. We provide experimental results to prove this.
A systematic study on parameter correlations in large%2Dscale duplicate document detection.
Under consideration for publication in Knowledge and Information
An active feedback framework for image retrieval.
Available online at www.sciencedirect.com
The Fourth Asian Information Retrieval Symposium %28AIRS 08%29.

Webpage understanding beyond page%2Dlevel search.

Advances in Multimedia Information Processing %2D PCM 2007

Scalable music recommendation by search.
The growth of music resources on personal devices and Internet radio has increased the need for music recommendations. In this paper, aiming at providing an efficient and general solution, we present a search-based solution for scalable music recommendations. In this solution a music piece is first transformed to a music signature sequence in which each signature characterizes the timbre of a local music clip. Based on such signatures, a scale-sensitive method is then proposed to index the music pieces for similarity search, using the locality sensitive hashing (LSH). The scale-sensitive method can numerically find the appropriate parameters for indexing various scales of music collections, and thus can guarantee a proper number of nearest neighbors are found in search. In the recommendation stage, representative signatures from snippets of a seed piece are extracted as query terms, to retrieve pieces with similar melodies for suggestions. We also design a relevance-ranking function to sort the search results, based on the criteria that include matching ratio, temporal order, term weight, andmatching confidence. Finally, with the search results, we propose a strategy to generate a dynamic playlist which can automatically expand with time. Evaluations of several music collections at various scales showed that our approach achieves encouraging results in terms of recommendation satisfaction and system scalability.
MusicSense contextual music recommendation using emotional allocation modeling.
In this paper, we present a novel contextual music recommendation approach, MusicSense, to automatically suggest music when users read Web documents such as Weblogs. MusicSense matches music to a document’s content, in terms of the emotions expressed by both the document and the music songs. To achieve this, we propose a generative model– Emotional Allocation Modeling–in which a collection of word terms is considered as generated with a mixture of emotions. This model also integrates knowledge discovering from a Web-scale corpus and guidance from psychological studies of emotion. Music songs are also described using textual information extracted from their meta-data and relevant Web pages. Thus, both music songs and Web documents can be characterized as distributions over the emotion mixtures through the emotional allocation modeling. For a given document, the songs with the most matched emotion distributions are finally selected as the recommendations. Preliminary experiments on Weblogs show promising results on both emotion allocation and music recommendation.
Bipartite graph reinforcement model for web image annotation.

Dual cross%2Dmedia relevance model for image annotation.

Object%2Dlevel Vertical Search.
The primary function of current Web search engines is essentially relevance ranking at the document level. However, myriad structured information about real-world objects is embedded in static Web pages and online Web databases. Document-level information retrieval can unfortunately lead to highly inaccurate relevance ranking in answering object-oriented queries. In this paper, we propose a paradigm shift to enable searching at the object level. In traditional information retrieval models, documents are taken as the retrieval units and the content of a document is considered reliable. However, this reliability assumption is no longer valid in the object retrieval context when multiple copies of information about the same object typically exist. These copies may be inconsistent because of diversity of Web site qualities and the limited performance of current information extraction techniques. If we simply combine the noisy and inaccurate attribute information extracted from different sources, we may not be able to achieve satisfactory retrieval performance. In this paper, we propose several language models for Web object retrieval, namely an unstructured object retrieval model, a structured object retrieval model, and a hybrid model with both structured and unstructured retrieval features. We test these models on a paper search engine and compare their performances. We conclude that the hybrid model is the superior by taking into account the extraction errors at varying levels.
Exploring LDA%2DBased Document Model for Geographic Information Retrieval.

Computing Geographical Serving Area Based on Search Logs and Website Categorization.

Fast Large%2DScale Spectral Clustering by Sequential Shrinkage Optimization.

Improve Ranking by Using Image Information.

Mining geographic knowledge using location aware topic model.

Distributed Architecture for Large Scale Image%2DBased Search.
In recent years, some computer vision algorithms such as SIFT (Scale Invariant Feature Transform) have been employed in image similarity match to perform image-based search applications. However, with the increasing scale of image databases, centralized image retrieval system no longer provide adequate prompt search. In this paper, we design a scalable distributed architecture, which is analog to web search engine, for efficient large-scale image retrieval. In our distributed architecture, images are partitioned to multiple servers and an index is built. Administrated by a controlling server, each distributed server matches query image in its own image sub-collection in parallel and returns the intermediate search results, a list of images similar to query image, to the controlling server for further re-ranking and merging. An evaluation of the results shows that our distributed architecture removes the limitation of a centralized image retrieval system. By performing reasonable indexing, merging and ranking strategies, the precision level of search is near to that performed on standalone retrieval systems indexing all images. 1.
Searching One Billion Web Images by Content Challenges and Opportunities.

Dual%2DSpace Pyramid Matching for Medical Image Classification.

Visual language modeling for image classification.



FRank a ranking method with fidelity loss.
Ranking problem is becoming important in many fields, especially in information retrieval (IR). Many machine learning techniques have been proposed for ranking problem, such as RankSVM, RankBoost, and RankNet. Among them, RankNet, which is based on a probabilistic ranking framework, is leading to promising results and has been applied to a commercial Web search engine. In this paper we conduct further study on the probabilistic ranking framework and provide a novel loss function named fidelity loss for measuring loss of ranking. The fidelity loss not only inherits effective properties of the probabilistic ranking framework in RankNet, but possesses new properties that are helpful for ranking. This includes the fidelity loss obtaining zero for each document pair, and having a finite upper bound that is necessary for conducting query-level normalization. We also propose an algorithm named FRank based on a generalized additive model for the sake of minimizing the fidelity loss and learning an effective ranking function. We evaluated the proposed algorithm for two datasets: TREC dataset and real Web search dataset. The experimental results show that the proposed FRank algorithm outperforms other learning-based ranking methods on both conventional IR problem and Web searching.
Webstudio building infrastructure for web data management.

Web object retrieval.
The primary function of current Web search engines is essentially relevance ranking at the document level. However, myriad structured information about real-world objects is embedded in static Web pages and online Web databases. Document-level information retrieval can unfortunately lead to highly inaccurate relevance ranking in answering object-oriented queries. In this paper, we propose a paradigm shift to enable searching at the object level. In traditional information retrieval models, documents are taken as the retrieval units and the content of a document is considered reliable. However, this reliability assumption is no longer valid in the object retrieval context when multiple copies of information about the same object typically exist. These copies may be inconsistent because of diversity of Web site qualities and the limited performance of current information extraction techniques. If we simply combine the noisy and inaccurate attribute information extracted from different sources, we may not be able to achieve satisfactory retrieval performance. In this paper, we propose several language models for Web object retrieval, namely an unstructured object retrieval model, a structured object retrieval model, and a hybrid model with both structured and unstructured retrieval features. We test these models on a paper search engine and compare their performances. We conclude that the hybrid model is the superior by taking into account the extraction errors at varying levels.
Topic distillation via sub%2Dsite retrieval.

A survey of content%2Dbased image retrieval with high%2Dlevel semantics.

Clustering and searching WWW images using link and page layout analysis.
Due to the rapid growth of the number of digital images on the Web, there is an increasing demand for effective and efficient method for organizing and retrieving the images available. This paper describes iFind, a system for cluster-ing and searching WWW images. By using a vision-based page segmentation algorithm, a web page is partitioned into blocks, and the textual and link information of an image can be accurately extracted from the block containing that image. The textual information is used for image representation. By extracting the page-to-block, block-to-image, block-to-page relationships through link structure and page layout analysis, we construct an image graph. Our method is less sensitive to noisy links than previous methods like PicASHOW, and hence the image graph can better reflect the semantic relationship between images. Using the notion of Markov Chain, we can infer the semantic de-grees of the images, i.e. ImageRanks, which characterize the importance of the images. The ImageRanks are com-bined with the relevance scores to produce the final ranking for image search. With the graph models, we can also use techniques from spectral graph theory for image clustering and embedding, or 2-D visualization. Some experi-mental results on 11.6 million images downloaded from the Web are provided in the paper.
EnjoyPhoto a vertical image search engine for enjoying high%2Dquality photos.
In this paper, we propose building a vertical image search engine called EnjoyPhoto that leverages rich metadata from various photo forum web sites to meet users ’ requirements for enjoying high-quality photos, which is virtually impossible in traditional image search engines. To solve the ranking problem when aggregating multiple photo forums, we propose a novel rank fusion algorithm that uses duplicate photos to normalize rating scores. To further improve user experiences in enjoying photos, we design an in-place image browsing interface, and compare it with several other interfaces in a user study. With rich metadata and rating information, more attractive user interfaces are enabled, including slideshow authoring and photo recommendations. We conducted experiments and user studies on a 2.5-million image database to evaluate the proposed rank fusion algorithm, investigate the rationale behind building a vertical image search engine, and study user interfaces and preferences for the purpose of enjoying high-quality photos. The experimental results demonstrate the effectiveness of the proposed ranking algorithm. The results also show that the 2.5-million high-quality image database in EnjoyPhoto performs comparably with Google’s 1billion image database for queries related to location, nature, and daily life categories. Finally, our results show that the in-place browsing interface—called Force-Transfer view—is much more convenient for users than traditional interfaces.
IGroup web image search results clustering.
In this paper, we propose, IGroup, an efficient and effective algorithm that organizes Web image search results into clusters. IGroup is different from all existing Web image search results clustering algorithms that only cluster the top few images using visual or textual features. Our proposed algorithm first identifies several query-related semantic clusters based on a key phrases extraction algorithm originally proposed for clustering general Web search results. Then, all the resulting images are separated and assigned to corresponding clusters. As a result, all the resulting images are organized into a clustering structure with semantic level. To make the best use of the clustering results, a new user interface (UI) is proposed. Different from existing Web image search interfaces, which show only a limited number of suggested query terms or representative image thumbnails of some clusters, the proposed interface displays both representative thumbnails and appropriate titles of semantically coherent image clusters. Comprehensive user studies have been completed to evaluate both the clustering algorithm and the new UI.
IGroup a web image search engine with semantic clustering of search results.

VirtualTour an online travel assistant based on high quality images.
With the popularity of both travel and Web, more and more people use online travel services to facilitate their travel activities or share their travel experiences. Considering that existing services emphasize more on the textual content with the pictorial content only as supplement, we propose the VirtualTour system. It is an online travel service dedicated on high quality images, which helps travelers plan their trip. The images of VirtualTour are from photo forum sites. They have rich and accurate metadata which could be used to extract geographic location information of them and assess the quality of them. A representative sights identification algorithm is also proposed to automatically identify the possible related sights of a region. Based on the map services that seamlessly integrated into the system, a wieldy UI is designed to support several useful features, e.g. query by map, location or path. Categories and Subject Descriptors H.3.7 [Information storage and retrieval]: Digital Libraries – collection, systems issues, user issues. H.5.3 [Information interfaces and presentation]: Group and Organization Interfaces – organizational design, Web-based interaction.
Image annotation by large%2Dscale content%2Dbased image retrieval.
Image annotation has been an active research topic in recent years due to its potentially large impact on both image understanding and Web image search. In this paper, we target at solving the automatic image annotation problem in a novel search and mining framework. Given an uncaptioned image, first in the search stage, we perform content-based image retrieval (CBIR) facilitated by high-dimensional indexing to find a set of visually similar images from a large-scale image database. The database consists of images crawled from the World Wide Web with rich annotations, e.g. titles and surrounding text. Then in the mining stage, a search result clustering technique is utilized to find most representative keywords from the annotations of the retrieved image subset.
Towards content%2Dbased relevance ranking for video search.

Discovering Authoritative News Sources and Top News Stories.

Ranking web objects from multiple communities.

Adaptive User Profile Model and Collaborative Filtering for Personalized News.

Ranking web objects from multiple communities.

A comparative study on classifying the functions of web page blocks.
In this paper, we study the problem of learning block classification models to estimate block functions. We distinguish general models, which are learned across multiple sites, and site-specific models, which are learned within individual sites. We further consider several factors that affect the learning process and model effectiveness. These factors include the layout features, the content features, the classifiers, and the term selection methods. We have empirically evaluated the performance of the models when the factors are varied. Our main results are that layout features do better than content features for learning both general and site-specific models.
Medical Image Annotation and Retrieval Using Visual Features.

MSRA Columbus at GeoCLEF 2006.

AnnoSearch Image Auto%2DAnnotation by Search.
Although it has been studied for several years by computer vision and machine learning communities, image annotation is still far from practical. In this paper, we present AnnoSearch, a novel way to annotate images using search and data mining technologies. Leveraging the Web-scale images, we solve this problem in two-steps: 1) searching for semantically and visually similar images on the Web, 2) and mining annotations from them. Firstly, at least one accurate keyword is required to enable text-based search for a set of semantically similar images. Then content-based search is performed on this set to retrieve visually similar images. At last, annotations are mined from the descriptions (titles, URLs and surrounding texts) of these images. It worth highlighting that to ensure the efficiency, high dimensional visual features are mapped to hash codes which significantly speed up the content-based search process. Our proposed approach enables annotating with unlimited vocabulary, which is impossible for all existing approaches. Experimental results on real web images show the effectiveness and efficiency of the proposed algorithm. 1.
Learning Distance Metrics with Contextual Constraints for Image Retrieval.
Relevant Component Analysis (RCA) has been proposed for learning distance metrics with contextual constraints for image retrieval. However, RCA has two important disadvantages. One is the lack of exploiting negative constraints which can also be informative, and the other is its incapability of capturing complex nonlinear relationships between data instances with the contextual information. In this paper, we propose two algorithms to overcome these two disadvantages, i.e., Discriminative Component Analysis (DCA) and Kernel DCA. Compared with other complicated methods for distance metric learning, our algorithms are rather simple to understand and very easy to solve. We evaluate the performance of our algorithms on image retrieval in which experimental results show that our algorithms are effective and promising in learning good quality distance metrics for image retrieval. 1
Ranking Web News Via Homepage Visual Layout and Cross%2DSite Voting.

Exploring URL Hit Priors for Web Search.

Fast Spectral Clustering of Data Using Sequential Matrix Compression.

Automated known problem diagnosis with event traces.
Computer problem diagnosis remains a serious challenge to users and support professionals. Traditional troubleshooting methods relying heavily on human intervention make the process inefficient and the results inaccurate even for solved problems, which contribute significantly to user’s dissatisfaction. We propose to use system behavior information such as system event traces to build correlations with solved problems, instead of using only vague text descriptions as in existing practices. The goal is to enable automatic identification of the root cause of a problem if it is a known one, which would further lead to its resolution. By applying statistical learning techniques to classifying system call sequences, we show our approach can achieve considerable accuracy of root cause recognition by studying four case examples.
Detecting Geographical Serving Area of Web Resources.

Indexing implicit locations for geographical information retrieval.

Extracting Objects from the Web.
Automatic identification and extraction of interesting objects instead of just related Web pages is useful to users of the World Wide Web. For example, an instructor teaching a Data Mining course may be interested in finding all text books, lecture notes, slides, homeworks and term projects from Web pages of all Data Mining and their related courses. In this paper, we present a tool WISE for extraction of objects from web pages, which are of interest. In WISE, objects in HTML pages are defined as nodes in an extension of DOM trees. Extraction of objects is then formulated as the problem of classification of tree nodes. We develop a mechanism for object specifications, and a specific-togeneral (bottom-up) search algorithm for learning object specifications. Experiment shows that WISE works well for extracting objects such as, for example, extracting items in paper lists, and extracting lists of lecture notes in Algorithms courses. 1
Query Selection Techniques for Efficient Crawling of Structured Web Sources.

Star%2DStructured High%2DOrder Heterogeneous Data Co%2Dclustering Based on Consistent Information Theory.

Using Implicit Relevane Feedback to Advance Web Image Search.

Inquiring of the Sights from the Web via Camera Mobiles.

Large%2DScale Duplicate Detection for Web Image Search.

Inquiring of the Sights from the Web via Camera Mobiles.

Automatic Classification of Photographs and Graphics.

Event detection from evolution of click%2Dthrough data.
Previous efforts on event detection from the web have focused primarily on web content and structure data ignoring the rich collection of web log data. In this paper, we propose the first approach to detect events from the click-through data, which is the log data of web search engines. The intuition behind event detection from click-through data is that such data is often event-driven and each event can be represented as a set of query-page pairs that are not only semantically similar but also have similar evolution pattern over time. Given the click-through data, in our proposed approach, we first segment it into a sequence of bipartite graphs based on the user-defined time granularity. Next, the sequence of bipartite graphs is represented as a vectorbased graph, which records the semantic and evolutionary relationships between queries and pages. After that, the vector-based graph is transformed into its dual graph, where each node is a query-page pair that will be used to represent real world events. Then, the problem of event detection is equivalent to the problem of clustering the dual graph of the vector-based graph. The clustering process is based on a two-phase graph cut algorithm. In the first phase, querypage pairs are clustered based on the semantic-based similarity such that each cluster in the result corresponds to a specific topic. In the second phase, query-page pairs related to the same topic are further clustered based on the evolution pattern-based similarity such that each cluster is expected to represent a specific event under the specific topic. Experiments with real click-through data collected from a commercial web search engine show that the proposed approach produces high quality results.

Simultaneous record detection and attribute labeling in web data extraction.
Recent work has shown the feasibility and promise of templateindependent Web data extraction. However, existing approaches use decoupled strategies – attempting to do data record detection and attribute labeling in two separate phases. In this paper, we show that separately extracting data records and attributes is highly ineffective and propose a probabilistic model to perform these two tasks simultaneously. In our approach, record detection can benefit from the availability of semantics required in attribute labeling and, at the same time, the accuracy of attribute labeling can be improved when data records are labeled in a collective manner. The proposed model is called Hierarchical Conditional Random Fields. It can efficiently integrate all useful features by learning their importance, and it can also incorporate hierarchical interactions which are very important for Web data extraction. We empirically compare the proposed model with existing decoupled approaches for product information extraction, and the results show significant improvements in both record detection and attribute labeling.
Detecting The Sufficient Display Resolution For Image Browsing.


Study on texture feature extraction in region%2Dbased image retrieval system.
Abstract image. In the right image, the &#039;sand &#039; region is usually what users are interested in. T&#039;exture is an important feature to describe images. 7hough lots of work has been done for efficient texture feature extraction from rectangular images, no much ef,fort has been made in texture feature extraction from arhitrarv-shaped regions in region-based image retrieval (RBIR) svstem. In this paper, we present an efficient texture feature extraction algorithm for arbitrary-shaped Figurel. Interested region in image regions. This algorithm first extends an arbitrary-shaped region inito a rectangutar area onto which block tran.sformiation can be applied. Based on the Projection-Onto-Convex-Sets (POCS) theory, a set of coefficients hest describing the original region are finally obtained,
An adaptive graph model for automatic image annotation.

Heterogeneous Information Integration in Hierarchical Text Classification.

Level%2DBiased Statistics in the Hierarchical Structure of the Web.

A Systematic Study of Parameter Correlations in Large Scale Duplicate Document Detection.
Under consideration for publication in Knowledge and Information
Level%2DBiased Statistics in the Hierarchical Structure of the Web.

Building implicit links from content for forum search.
The objective of Web forums is to create a shared space for open communications and discussions of specific topics and issues. The tremendous information behind forum sites is not fully-utilized yet. Most links between forum pages are automatically created, which means the link-based ranking algorithm cannot be applied efficiently. In this paper, we proposed a novel ranking algorithm which tries to introduce the content information into link-based methods as implicit links. The basic idea is derived from the more focused random surfer: the surfer may more likely jump to a page which is similar to what he is reading currently. In this manner, we are allowed to introduce the content similarities into the link graph as a personalization bias. Our method, named Finegrained Rank (FGRank), can be efficiently computed based on an automatically generated topic hierarchy. Not like the topic-sensitive PageRank, our method only need to compute single PageRank score for each page. Another contribution of this paper is to present a very efficient algorithm for automatically generating topic hierarchy and map each page in a large-scale collection onto the computed hierarchy. The experimental results show that the proposed method can improve retrieval performance, and reveal that content-based link graph is also important compared with the hyper-link graph.
AggregateRank bringing order to web sites.

Image annotation using search and mining technologies.

Time%2Ddependent semantic similarity measure of queries using historical click%2Dthrough data.
It has become a promising direction to measure similarity of Web search queries by mining the increasing amount of clickthrough data logged by Web search engines, which record the interactions between users and the search engines. Most existing approaches employ the click-through data for similarity measure of queries with little consideration of the temporal factor, while the click-through data is often dynamic and contains rich temporal information. In this paper we present a new framework of time-dependent query semantic similarity model on exploiting the temporal characteristics of historical click-through data. The intuition is that more accurate semantic similarity values between queries can be obtained by taking into account the timestamps of the log data. With a set of user-defined calendar schema and calendar patterns, our time-dependent query similarity model is constructed using the marginalized kernel technique, which can exploit both explicit similarity and implicit semantics from the click-through data effectively. Experimental results on a large set of click-through data acquired from a commercial search engine show that our time-dependent query similarity model is more accurate than the existing approaches. Moreover, we observe that our time-dependent query similarity model can, to some extent, reflect real-world semantics such as real-world events that are happening over time.
Image Search Engine.
PAMI, 20(7):751 -- 756, 1998.  [168] P. M. Tardif and A. Zaccarin. Multiscale autoregressive image representation for texture segmentation. In Proceedings of SPIE Vol. 3026, Nonlinear Image Processing VIII, San Jose, CA, USA, pages 327--337, 1997.  [169] K. Tieu and P. Viola. Boosting image retrieval. In Computer Vision and Pattern Recognition, pages 228--235, 2000.  [170] T. Tuytelaars and L. van Gool. Content-based image retrieval based on local a#nely invariant regions. In Proceedings of Visual Information and Information Systems, pages 493 -- 500, 1999.  [171] A. Vailaya, M. Figueiredo, A. Jain, and H. Zhang. A bayesian framework for semantic classification of outdoor vacation images. In C. a. Bouman M. M. Yeung, B. Yeo, editor, Storage and Retrieval for Image and Video Databases VII - SPIE, pages 415--426, 1999.  [172] A. Vailaya, M. Figueiredo, A. Jain, and H. Zhang. Content-based hierarchical classification of vacation images. In IEEE International Conference on Multimedia Compu
Multitype Features Coselection for Web Document Clustering.

Design and Performance Studies of an Adaptive Scheme for Serving Dynamic Web Content in a Mobile Computing Environment.

Browsing Large Pictures Under Limited Display Sizes.
 Pictures have become increasingly common and popular in mobile communications. However, due to the limitation of mobile devices, there is a need to develop new technologies to facilitate the browsing of large pictures on the small screen. In this paper, we propose a set of novel approaches which are able to aid or automate common image browsing tasks on mobile devices. All of these approaches are based on an image attention model which is employed to illustrate the information structure within an image. An efficient algorithm to generate the optimal browsing path based on the information foraging theory is presented. Experimental evaluations of the proposed mechanism indicate that our approach is an effective way for viewing large images on small displays.  
A scalable supervised algorithm for dimensionality reduction on streaming data.

Exploring statistical correlations for image retrieval.
Bridging the cognitive gap in image retrieval has been an active research direction in recent years, of which a key challenge is to get enough training data to learn the mapping functions from low-level feature spaces to high-level semantics. In this paper, image regions are classified into two types: key regions representing the main semantic contents and environmental regions representing the contexts. We attempt to leverage the correlations between types of regions to improve the performance of image retrieval. A Context Expansion approach is explored to take advantages of such correlations by expanding the key regions of the queries using highly correlated environmental regions according to an image thesaurus. The thesaurus serves as both a mapping function between image low-level features and concepts and a store of the statistical correlations between different concepts. It is constructed through a data-driven approach which uses Web data (images, their surrounding textual annotations) as training data source to learn the region concepts and to explore the statistical correlations. Experimental results on a database of 10,000 general-purpose images show the effectiveness of our proposed approach in both improving search precision (i.e. filter irrelevant images) and recall (i.e. retrieval relevant images whose context may be varied). Several major factors which have impact on the performance of our approach are also studied. Key words: query expansion, image thesaurus, content-based image retrieval, region-based image retrieval
A probabilistic semantic model for image annotation and multi%2Dmodal image retrieval.
This paper addresses automatic image annotation problem and its application to multi-modal image retrieval. The contribution of our work is three-fold. (1) We propose a probabilistic semantic model in which the visual features and the textual words are connected via a hidden layer which constitutes the semantic concepts to be discovered to explicitly exploit the synergy among the modalities. (2) The association of visual features and textual words is determined in a Bayesian framework such that the confidence of the association can be provided. (3) Extensive evaluation on a large-scale, visually and semantically diverse image collection crawled from Web is reported to evaluate the prototype system based on the model. In the proposed probabilistic model, a hidden concept layer which connects the visual feature and the word layer is discovered by fitting a generative model to the training image and annotation words through an Expectation-Maximization (EM) based iterative learning procedure. The evaluation of the prototype system on 17,000 images and 7,736 automatically extracted annotation words from crawled Web pages for multi-modal image retrieval has indicated that the proposed semantic model and the developed Bayesian framework are superior to a state-of-the-art peer system in the literature. 1.
TSSP Multi%2Dfeatures based reinforcement algorithm to find related papers.

Image and Video Retrieval

Web image clustering by consistent utilization of visual features and surrounding texts.
Image clustering, an important technology for image processing, has been actively researched for a long period of time. Especially in recent years, with the explosive growth of the Web, image clustering has even been a critical technology to help users digest the large amount of online visual information. However, as far as we know, many previous works on image clustering only used either low-level visual features or surrounding texts, but rarely exploited these two kinds of information in the same framework. To tackle this problem, we proposed a novel method named consistent bipartite graph co-partitioning in this paper, which can cluster Web images based on the consistent fusion of the information contained in both low-level features and surrounding texts. In particular, we formulated it as a constrained multiobjective optimization problem, which can be efficiently solved by semi-definite programming (SDP). Experiments on a realworld Web image collection showed that our proposed method outperformed the methods only based on low-level features or surround texts.
Iteratively clustering web images based on link and attribute reinforcements.
Image clustering is an important research topic which contributes to a wide range of applications. Traditional image clustering approaches are based on image content features only, while content features alone can hardly describe the semantics of the images. In the context of Web, images are no longer assumed homogeneous and “flat ” distributed but are richly structured. There are two kinds of reinforcement embedded in such data: 1) the reinforcement between attributes of different data types (intraintra reinforcements); and 2) the reinforcement between object attributes and the inter-type links (intra-inter reinforcements). Unfortunately, most of the previous works addressing relational data failed to fully explore the reinforcements. In this paper, we propose a reinforcement clustering framework to tackle this problem. It reinforces images and texts ’ attributes via inter-type links and inversely uses these attributes to update these links. The iterative reinforcing nature of this framework promises the discovery of the semantic structure of images, which is the basis of image clustering. Experimental results show the effectiveness of our proposed framework.
Graph based multi%2Dmodality learning.
To better understand the content of multimedia, a lot of research efforts have been made on how to learn from multi-modal feature. In this paper, it is studied from a graph point of view: each kind of feature from one modality is represented as one independent graph; and the learning task is formulated as inferring from the constraints in every graph as well as supervision information (if available). For semi-supervised learning, two different fusion schemes, namely linear form and sequential form, are proposed. For each scheme, it is derived from optimization point of view; and further justified from two sides: similarity propagation and Bayesian interpretation. By doing so, we reveal the regular optimization nature, transductive learning nature as well as prior fusion nature of the proposed schemes, respectively. Moreover, the proposed method can be easily extended to unsupervised learning, including clustering and embedding. Systematic experimental results validate the effectiveness of the proposed method.
Subsite Retrieval A Novel Concept for Topic Distillation.
Abstract. Topic distillation is one of the main information needs when users search the Web. In previous approaches to topic distillation, the single page was treated as the basic searching unit. This strategy is inherited from general information retrieval, which has not fully utilized the structure information of the Web. In this paper, we propose a novel concept for topic distillation, named subsite retrieval, in which the basic searching unit is the subsite instead of the single page. As indicated by the name, the subsite is a subset of website, consisting of a structural collection of pages. The key of subsite retrieval is to extract effective features to represent a subsite by utilizing both the content in each page and the structural information in the subsite. Specifically, we propose a so-called PI algorithm for this purpose, which is based on the modeling of website growth. Testing on the topic distillation task of TREC 2003 and TREC 2004, subsite retrieval gets significant improvement of retrieval performance over the previous single page based methods. 1
Calculating Webpage Importance with Site Structure Constraints.
Abstract. PageRank is one of the most popular link analysis algorithms that have shown their effectiveness in web search. However, PageRank only consider hyperlink information. In this paper, we propose several novel ranking algorithms, which make use of both hyperlink and site structure information to measure the importance of each web page. Specifically, two kinds of methodologies are adopted to refine the PageRank algorithm: one combines hyperlink information and website structure information together by graph fusion to refine PageRank algorithm, while the other re-ranks the pages within the same site by quadratic optimization based on original PageRank values. Experiments show that both two methodologies effectively improve the retrieval performance. 1
KNM A Novel Intelligent User Interface for Webpage Navigation.

A Similarity Reinforcement Algorithm for Heterogeneous Web Pages.

Level%2DBased Link Analysis.
Abstract. In order to get high-quality web pages, search engines often resort retrieval pages by their ranks. The rank is a kind of measurement of importance of pages. Famous ranking algorithms, including PageRank and HITS, make use of hyperlinks to compute the importance. Those algorithms consider all hyperlinks identically in sense of recommendation. However, we find that the World Wide Web is actually organized with the natural multi-level structure. Benefiting from the level information of pages, we can describe the recommendation of hyperlinks more reasonably and precisely. With this motivation, a new level-based link analysis algorithm is proposed in this paper. In the proposed algorithm, the recommendation weight of each hyperlink is computed with the level properties of its two endings. Experiments on the topic distillation task of TREC2003 web track show that our algorithm can evidently improve searching results as compared to previous link analysis methods. 1
Supervised Semi%2Ddefinite Embedding for Email Data Cleaning and Visualization.

Learning user interest for image browsing on small%2Dform%2Dfactor devices.
Mobile devices which can capture and view pictures are becoming increasingly common in our life. The limitation of these small-form-factor devices makes the user experience of image browsing quite different from that on desktop PCs. In this paper, we first present a user study on how users interact with a mobile image browser with basic functions. We found that on small displays, users tend to use more zooming and scrolling actions in order to view interesting regions in detail. From this fact, we designed a new method to detect user interest maps and extract user attention objects from the image browsing log. This approach is more efficient than image-analysis based methods and can better represent users ’ actual interest. A smart image viewer was then developed based on user interest analysis. A second experiment was carried out to study how users behave with such a viewer. Experimental results demonstrate that the new smart features can improve the browsing efficiency and are a good compliment to traditional image browsers.
Hybrid index structures for location%2Dbased web search.

A Unified Optimization Based Learning Method for Image Retrieval.
In this paper, an optimization based learning method is proposed for image retrieval from graph model point of view. Firstly, image retrieval is formulated as a regularized optimization problem, which simultaneously considers the constraints from low-level feature, online relevance feedback and offline semantic information. Then, the global optimal solution is developed in both closed form and iterative form, providing that the latter converges to the former. The proposed method is unified in the senses that 1) it makes use of the information from various aspects in a global optimization manner so that the retrieval performance might be maximally improved; 2) it provides a natural way to support two typical query scenarios in image retrieval. The proposed method has a solid mathematical ground. Systematic experimental results on a general-purpose image database demonstrate that it achieves significant improvements over existing methods.
Detecting geographic locations from web resources.

A Probabilistic Semantic Model for Image Annotation and Multi%2DModal Image Retrieva.

Natural Image Retrieval with Sketches.

Auto cropping for digital photographs.
In this paper, we propose an effective approach to the nearly untouched problem, still photograph auto cropping, which is one of the important features to automatically enhance photographs. To obtain an optimal result, we first formulate auto cropping as an optimization problem by defining an energy function, which consists of three sub models: composition sub model, conservative sub model and penalty sub model. Then, particle swarm optimization (PSO) is employed to obtain the optimal solution by maximizing the objective function. Experimental results and user studies over hundreds of photographs show that the proposed approach is effective and accurate in most cases, and can be used in many practical multimedia applications. 1.
Supervised semi%2Ddefinite embedding for image manifolds.

Automatic Annotation of Location Information for WWW Images.

2D Conditional Random Fields for Web information extraction.
The Web contains an abundance of useful semistructured information about real world objects, and our empirical study shows that strong sequence characteristics exist for Web information about objects of the same type across different Web sites. Conditional Random Fields (CRFs) are the state of the art approaches taking the sequence characteristics to do better labeling. However, as the information on a Web page is two-dimensionally laid out, previous linear-chain CRFs have their limitations for Web information extraction. To better incorporate the two-dimensional neighborhood interactions, this paper presents a two-dimensional CRF model to automatically extract object information from the Web. We empirically compare the proposed model with existing linear-chain CRF models for product information extraction, and the results show the effectiveness of our model. 1.
Web object indexing using domain knowledge.

Consistent bipartite graph co%2Dpartitioning for star%2Dstructured high%2Dorder heterogeneous data co%2Dclustering.
Heterogeneous data co-clustering has attracted more and more attention in recent years due to its high impact on various applications. While the co-clustering algorithms for two types of heterogeneous data (denoted by pair-wise co-clustering), such as documents and terms, have been well studied in the literature, the work on more types of heterogeneous data (denoted by high-order co-clustering) is still very limited. As an attempt in this direction, in this paper, we worked on a specific case of high-order coclustering in which there is a central type of objects that connects the other types so as to form a star structure of the interrelationships. Actually, this case could be a very good abstract for many real-world applications, such as the co-clustering of categories, documents and terms in text mining. In our philosophy, we treated such kind of problems as the fusion of multiple pairwise co-clustering sub-problems with the constraint of the star structure. Accordingly, we proposed the concept of consistent bipartite graph co-partitioning, and developed an algorithm based on semi-definite programming (SDP) for efficient computation of the clustering results. Experiments on toy problems and real data both verified the effectiveness of our proposed method.
Sports Video Mining with Mosaic.
Video is an information-intensive media with much redundancy. Therefore, it is desirable to be able to mine structure or semantics of video data for efficient browsing, summarization and highlight extraction. In this paper, we propose a generic approach to keyevent as well as structure mining for sports video analysis. Mosaic is generated for each shot as the representative image of shot content. Based on mosaic, sports video is mined by the method with prior knowledge and without prior knowledge. Without prior knowledge, our system may locate plays by discriminating those segments without essential content, such as breaks. If prior knowledge is available, the key-events in plays are detected using robust features extracted from mosaic. Experimental results have demonstrated the effectiveness and robustness of this sports video mining approach. 1.
Realistic 3D Face Modeling by Fusing Multiple 2D Images.
In this paper, we propose a fully automatic and efficient algorithm for realistic 3D face reconstruction by fusing multiple 2D face images. Firstly, an efficient multiview 2D face alignment algorithm is utilized to localize the facial points of the face images; and then the intrinsic shape and texture models are inferred by the proposed Syncretized Shape Model (SSM) and Syncretized Texture Model (STM), respectively. Compared with other related works, our proposed algorithm has the following characteristics: 1�the inferred shape and texture are more realistic owing to the constraints and co-enhancement among the multiple images; 2) it is fully automatic, without any user interaction; and 3) the shape and pose parameter estimation is efficient via EM approach and unit quaternion based pose representation, and is also robust as a result of the dynamic correspondence approach. The experimental results show the effectiveness of our proposed algorithm for 3D face reconstruction.
Effective Feature Extraction for Play Detection in American Football Video.

Subspace Clustering and Label Propagation for Active Feedback in Image Retrieval.

Region%2DBased Image Retrieval with High%2DLevel Semantic Color Names.
Performance of traditional content-based image retrieval systems is far from user’s expectation due to the ‘semantic gap ’ between low-level visual features and the richness of human semantics. In attempt to reduce the ‘semantic gap’, this paper introduces a region-based image retrieval system with high-level semantic color names. In this system, database images are segmented into color-texture homogeneous regions. For each region, we define a color name as that used in our daily life. In the retrieval process, images containing regions of same color name as that of the query are selected as candidates. These candidate images are further ranked based on their color and texture features. In this way, the system reduces the ‘semantic gap ’ between numerical image features and the rich semantics in the user’s mind. Experimental results show that the proposed system provides promising retrieval results with few features used. 1.
ASAP A Synchronous Approach for Photo Sharing across Multiple Devices.

Parallel Image Matrix Compression for Face Recognition.
The canonical face recognition algorithm Eigenface and Fisherface are both based on one dimensional vector representation. However, with the high feature dimensions and the small training data, face recognition often suffers from the curse of dimension and the small sample problem. Recent research [4] shows that face recognition based on direct 2D matrix representation, i.e. 2DPCA, obtains better performance than that based on traditional vector representation. However, there are three questions left unresolved in the 2DPCA algorithm: 1) what is the meaning of the eigenvalue and eigenvector of the covariance matrix in 2DPCA; 2) why 2DPCA can outperform Eigenface; and 3) how to reduce the dimension after 2DPCA directly. In this paper, we analyze 2DPCA in a different view and proof that 2DPCA is actually a “localized” PCA with each row vector of an image as object. With this explanation, we discover the intrinsic reason that 2DPCA can outperform Eigenface is because fewer feature dimensions and more samples are used in 2DPCA, when compared with Eigenface. To further reduce the dimension after 2DPCA, a two-stage strategy, namely parallel image matrix compression (PIMC), is proposed to compress the image matrix redundancy, which exists among row vectors and column vectors. The exhaustive experiment results demonstrate that PIMC is superior to 2DPCA and Eigenface, and PIMC+LDA outperforms 2DPCA+LDA and Fisherface. 1.
Learning No%2DReference Quality Metric by Examples.
In this paper, a novel learning based method is proposed for No-Reference image quality assessment. Instead of examining the exact prior knowledge for the given type of distortion and finding a suitable way to represent it, our method aims to directly get the quality metric by means of learning. At first, some training examples are prepared for both high-quality and lowquality classes; then a binary classifier is built on the training set; finally the quality metric of an un-labeled example is denoted by the extent to which it belongs to these two classes. Different schemes to acquire examples from a given image, to build the binary classifier and to model the quality metric are proposed and investigated. While most existing methods are tailored for some specific distortion type, the proposed method might provide a general solution for No-Reference image quality assessment. Experimental results on JPEG and JPEG2000 compressed images validate the effectiveness of the proposed method. 1.
Grouping WWW Image Search Results by Novel Inhomogeneous Clustering Method.

AIRE an ambient interactive and responsive environment for mobile image management.

From relevance to intelligence toward next generation web search.

Photo%2Dto%2Dsearch using multimodal queries to search the web from mobile devices.

Multiple random walk and its application in content%2Dbased image retrieval.

Multimedia information retrieval what is it%2C and why isn%27t anyone using it%3F
In this paper, the participants of the panel at the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval answer questions about what multimedia is, how MIR is different from other kinds of retrieval, the most important technical challenges in MIR, killer applications, opportunities, and future directions.
Similarity space projection for web image search and annotation.
Web image search has been explored and developed in academic as well as commercial areas for over a decade. To measure the similarity between Web images and user queries, most of the existing Web image search systems try to convert an image to textual keywords by analyzing the textual information available (such as surrounding text and image filename) with or without leveraging image visual features (such as color, texture, shape). In this way, the existing systems transform “Web images ” to the “query (text) ” space so as to compare the relevance of images to the query. In this paper, we present a novel solution to Web image search-similarity space projection (SSP). This algorithm takes images and queries as two heterogeneous object peers, and projects them into a third Euclidean “similarity space ” in which their similarity can be directly measured. The rule of projection guarantees that in the new space the relevant images are kept close to the corresponding query and those irrelevant ones are away from it. Experiments on realworld Web image collections showed that the proposed algorithm significantly outperformed traditional information retrieval models (such as vector space model) in the application of image search. Besides Web image search, we demonstrate that this algorithm can also be applied to image annotation scenario, and has promising performance. Thus, this algorithm unifies Web image search and image annotation into same framework.
Multi%2Dgraph enabled active learning for multimodal web image retrieval.

Salient Feature Selection for Visual Concept Learning.
Abstract. Image classification could be treated as an effective solution to enable keyword-based semantic image retrieval. In this paper, we propose a novel image classification framework by learning semantic concepts of image categories. To choose representative features for an image category and meanwhile reduce noisy features, a three-step salient feature selection strategy is proposed. In the feature selection stage, salient patches are first detected and clustered. Then the region of dominance and salient entropy measures are calculated to reduce non-common
Efficient Browsing of Web Search Results on Mobile Devices Based on Block Importance Model.
It is expected that more and more people will search the web when they are on the move. Though conventional search engines can be directly visited from mobile devices with web browsing capabilities, the information is not as conveniently accessible from a handheld device as it is from desktops. Existing information discovery mechanisms for searching the web are not well-suited to mobile devices. In this paper, a block importance model is employed to assign importance values to different segments of a web page, in order to extract and present more condensed search results to mobile users. Based on the block importance model, three presentations for displaying the result pages in different levels of detail have been designed to reduce both the number of user interactions and the overall search time. A set of user study experiments have been carried out to compare the three presentations and a commercial service on typical mobile devices. Experimental results show that our approaches can help users to explore web search results more efficiently. 1.
A probabilistic model for retrospective news event detection.
Retrospective news event detection (RED) is defined as the discovery of previously unidentified events in historical news corpus. Although both the contents and time information of news articles are helpful to RED, most researches focus on the utilization of the contents of news articles. Few research works have been carried out on finding better usages of time information. In this paper, we do some explorations on both directions based on the following two characteristics of news articles. On the one hand, news articles are always aroused by events; on the other hand, similar articles reporting the same event often redundantly appear on many news sources. The former hints a generative model of news articles, and the latter provides data enriched environments to perform RED. With consideration of these characteristics, we propose a probabilistic model to incorporate both content and time information in a unified framework. This model gives new representations of both news articles and news events. Furthermore, based on this approach, we build an interactive RED system, HISCOVERY, which provides additional functions to present events, Photo Story and Chronicle.
OCFS optimal orthogonal centroid feature selection for text categorization.
ABSTRACT 1 Text categorization is an important research area in many Information Retrieval (IR) applications. To save the storage space and computation time in text categorization, efficient and effective algorithms for reducing the data before analysis are highly desired. Traditional techniques for this purpose can generally be classified into feature extraction and feature selection. Because of efficiency, the latter is more suitable for text data such as web documents. However, many popular feature selection techniques such as Information Gain (IG) and 2 χ-test (CHI) are all greedy in nature and thus may not be optimal according to some criterion. Moreover, the performance of these greedy methods may be deteriorated when the reserved data dimension is extremely low. In this paper, we propose an efficient optimal feature selection algorithm by optimizing the objective function of Orthogonal Centroid (OC) subspace learning algorithm in a discrete solution space, called Orthogonal Centroid Feature Selection (OCFS). Experiments on 20 Newsgroups (20NG), Reuters Corpus Volume 1 (RCV1) and Open Directory Project (ODP) data show that OCFS is consistently better than IG and CHI with smaller computation time especially when the reduced dimension is extremely small.
A study of relevance propagation for web search.
Different from traditional information retrieval, both content and structure are critical to the success of Web information retrieval. In recent years, many relevance propagation techniques have been proposed to propagate content information between web pages through web structure to improve the performance of web search. In this paper, we first propose a generic relevance propagation framework, and then provide a comparison study on the effectiveness and efficiency of various representative propagation models that can be derived from this generic framework. We come to many conclusions that are useful for selecting a propagation model in real-world search applications, including 1) sitemapbased propagation models outperform hyperlink-based models in sense of both effectiveness and efficiency, and 2) sitemap-based term propagation is easier to be integrated into real-world search engines because of its parallel offline implementation and acceptable complexity. Some other more detailed study results are also reported in the paper.
Detecting dominant locations from search queries.

Gravitation%2Dbased model for information retrieval.
This paper proposes GBM (gravitation-based model), a physical model for information retrieval inspired by Newton’s theory of gravitation. A mapping is built in this model from concepts of information retrieval (documents, queries, relevance, etc) to those of physics (mass, distance, radius, attractive force, etc). This model actually provides a new perspective on IR problems. A family of effective term weighting functions can be derived from it, including the well-known BM25 formula. This model has some advantages over most existing ones: First, because it is directly based on basic physical laws, the derived formulas and algorithms can have their exp licit physical interpretation. Second, the ranking formulas derived from this model satisfy more intuitive heuristics than most of existing ones, thus have the potential to behave empirically better and to be used safely on various settings. Finally, a new approach for structured document retrieval derived from this model is more reasonable and behaves better than existing ones.
Improving web search results using affinity graph.
In this paper, we propose a novel ranking scheme named Affinity Ranking (AR) to re-rank search results by optimizing two metrics: (1) diversity-- which indicates the variance of topics in a group of documents; (2) information richness-- which measures the coverage of a single document to its topic. Both of the two metrics are calculated from a directed link graph named Affinity Graph (AG). AG models the structure of a group of documents based on the asymmetric content similarities between each pair of documents. Experimental results in Yahoo! Directory, ODP Data, and Newsgroup data demonstrate that our proposed ranking algorithm significantly improves the search performance. Specifically, the algorithm achieves 31 % improvement in diversity and 12 % improvement in information richness relatively within the top 10 search results.
Object%2Dlevel ranking bringing order to Web objects.
In contrast with the current Web search methods that essentially do document-level ranking and retrieval, we are exploring a new paradigm to enable Web search at the object level. We collect Web information for objects relevant for a specific application domain and rank these objects in terms of their relevance and popularity to answer user queries. Traditional PageRank model is no longer valid for object popularity calculation because of the existence of heterogeneous relationships between objects. This paper introduces PopRank, a domain-independent object-level link analysis model to rank the objects within a specific domain. Specifically we assign a popularity propagation factor to each type of object relationship, study how di#erent popularity propagation factors for these heterogeneous relationships could affect the popularity ranking, and propose e#cient approaches to automatically decide these factors. Our experiments are done using 1 million CS papers, and the experimental results show that PopRank can achieve significantly better ranking results than naively applying PageRank on the object graph.
An experimental study on large%2Dscale web categorization.
Taxonomies of the Web typically have hundreds of thousands of categories and skewed category distribution over documents. It is not clear whether existing text classification technologies can perform well on and scale up to such large-scale applications. To understand this, we conducted the evaluation of several representative methods (Support Vector Machines, k-Nearest Neighbor and Naive Bayes) with Yahoo! taxonomies. In particular, we evaluated the effectiveness/efficiency tradeoff in classifiers with hierarchical setting compared to conventional (flat) setting, and tested popular threshold tuning strategies for their scalability and accuracy in large-scale classification problems.
Site abstraction for rare category classification in large%2Dscale web directory.

Web resource geographic location classification and detection.
Rapid pervasion of the web into users&#039; daily lives has put much importance on capturing location-specific information on the web, due to the fact that most human activities occur locally around where a user is located. This is especially true in the increasingly popular mobile and local search environments. Thus, how to correctly and effectively detect locations from web resources has become a key challenge to location-based web applications. In this paper, we first explicitly distinguish the locations of web resources into three types to cater to different application needs: 1) provider location; 2) content location; and 3) serving location. Then we describe a unified system that computes each of the three locations,  employing a set of algorithms and different geographic sources.
Representing personal web information using a topic%2Doriented interface.

Mobile multimedia services.
Personal mobility is one of the goals of Universal Personal Telecommunications (UPT) being specified  for future deployment. Most current efforts focus on telephony, with SS7 signaling. However, many  of the same goals can be accomplished for multimedia services, by using existing Internet protocols. We  describe a multimedia call/conference setup protocol that provides personal videophone addresses, independent  of the workstation a called party might be using at the time. The system is set up to use the  existing Internet email address as a videophone address. Location and call handling information is kept  at the subscriber&#039;s home site for improved access and privacy.  1 Introduction  The use of multimedia has progressed from stand-alone applications, to point-to-point, mostly local applications such as video-on-demand or LAN-based video conferencing. Across the wide-area, low-bandwidth circuit-switched teleconferencing (such as those using the H.320 standard) are spreading, as well...
An Editor Labeling Model for Training Set Expansion in Web Categorization.

Webpage Importance Analysis Using Conditional Markov Random Walk.

Adapting Web Pages for Small%2DScreen Devices.
As users get accustomed to an increasing diverse range of mobile devices, personal computers, intelligent home and office appliances, as well as shared public devices, collaborative multi-device systems are emerging to benefit from using together devices of different users and different capabilities. This paper presents a conceptual framework for the systematic design of such systems in symbiotic environments. It distinguishes between information and interaction spaces. Focusing on using devices together, we introduce relationships between interaction spaces and annotatable mappings between the information space and the interaction spaces. Customizable annotations and the corresponding transformation of information support the development of collaborative systems for different scenarios. To demonstrate the validity of our framework, we introduce an architecture for multi-device web-browsing systems, and an annotation vocabulary which particularly focuses on symbiotic environments. The evaluation of our implemented prototype confirms the usefulness of multi-browsing, and the need for supporting both developers and end users to individually annotate web pages.
Hierarchical Taxonomy Preparation for Text Categorization Using Consistent Bipartite Spectral Graph Copartitioning.

An adaptive web page layout structure for small devices.

Support vector machines classification with a very large%2Dscale taxonomy.
Very large-scale classification taxonomies typically have hundreds
Learning an image manifold for retrieval.
We consider the problem of learning a mapping function from low-level feature space to high-level semantic space. Under the assumption that the data lie on a submanifold embedded in a high dimensional Euclidean space, we propose a relevance feedback scheme which is naturally conducted only on the image manifold in question rather than the total ambient space. While images are typically represented by feature vectors in R n, the natural distance is often different from the distance induced by the ambient space R n. The geodesic distances on manifold are used to measure the similarities between images. However, when the number of data points is small, it is hard to discover the intrinsic manifold structure. Based on user interactions in a relevance feedback driven query-by-example system, the intrinsic similarities between images can be accurately estimated. We then develop an algorithmic framework to approximate the optimal mapping function by a Radial Basis Function (RBF) neural network. The semantics of a new image can be inferred by the RBF neural network. Experimental results show that our approach is effective in improving the performance of content-based image retrieval systems.
Grouping web image search result.

Efficient propagation for face annotation in family albums.
In this paper, we propose and investigate a new user scenario for face annotation, in which users are allowed to multi-select a group of photographs and assign names to these photographs. The system will then attempt to propagate names from photograph level to face level, i.e. to infer the correspondence between name and face. Given the face similarity measure which combines methodologies from face recognition and content-based image retrieval, we formulate name propagation as an optimization problem. We define the objective function as the sum of similarities between each pair of faces of the same individual in different photographs, and propose an iterative optimization algorithm to infer the optimal correspondence. To make the propagation result reliable, a reject scheme is adopted to reject those with low confidence scores. Furthermore, we investigate the combination and alternation of browsing mode for propagation and viewer mode for annotation, so that each mode can benefit from additional inputs from the other mode. The experimental evaluation has been conducted within a typical family album of over one thousand photographs and the results show that the proposed approach is effective and efficient in automated face annotation in family albums.
Intuitive and effective interfaces for WWW image search engines.

Locality preserving clustering for image database.
It is important and challenging to make the growing image repositories easy to search and browse. Image clustering is a technique that helps in several ways, including image data preprocessing, user interface designing, and search result representation. Spectral clustering method has been one of the most promising clustering methods in the last few years, because it can cluster data with complex structure, and the (near) global optimum is guaranteed. However, existing spectral clustering algorithms, like Normalized Cut, are difficult to handle data points out of training set. In this paper, we propose a clustering algorithm named Locality Preserving Clustering (LPC), which shares many of the data representation properties of nonlinear spectral method. Yet LPC provides an explicit mapping function which is defined everywhere, both on training data points and testing points. Experimental results show that LPC is more accurate than both “direct Kmeans ” and “PCA + Kmeans”. We also show that LPC produces in general comparable results with Normalized Cut, yet is more efficient than Normalized Cut.
Multi%2Dmodel similarity propagation and its application for web image retrieval.

Hierarchical clustering of WWW image search results using visual
We consider the problem of clustering Web image search results. Generally, the image search results returned by an image search engine contain multiple topics. Organizing the results into different semantic clusters facilitates users ’ browsing. In this paper, we propose a hierarchical clustering method using visual, textual and link analysis. By using a vision-based page segmentation algorithm, a web page is partitioned into blocks, and the textual and link information of an image can be accurately extracted from the block containing that image. By using block-level link analysis techniques, an image graph can be constructed. We then apply spectral techniques to find a Euclidean embedding of the images which respects the graph structure. Thus for each image, we have three kinds of representations, i.e. visual feature based representation, textual feature based representation and graph based representation. Using spectral clustering techniques, we can cluster the search results into different semantic clusters. An image search example illustrates the potential of these techniques.
Efficient PageRank with Same Out%2DLink Groups.

Clustering%2DBased Navigation of Image Search Results on Mobile Devices.

A Query%2DDependent Duplicate Detection Approach for Large Scale Search Engines.
Abstract. Duplication of Web pages greatly hurts the perceived relevance of a search engine. Existing methods for detecting duplicated Web pages can be classified into two categories, i.e. offline and online methods. The offline methods target to detect all duplicates in a large set of Web pages, but none of the reported methods is capable of processing more than 30 million Web pages, which is about 1 % of the pages indexed by todays commercial search engines. On the contrary, the online methods focus on removing duplicated pages in the search results at run time. Although the number of pages to be processed is smaller, these methods could heavily increase the response time of search engines. Our experiments on real query logs show that there is a significant difference between popular and unpopular queries in terms of query number and duplicate distributions. Then, we propose a hybrid query-dependent duplicate detection method which combines both advantage of offline and online methods. This hybrid method provides not only an effective but also scalable solution for duplicate detection. 1
Optimizing web search using web click%2Dthrough data.
Search System
MRSSA an iterative algorithm for similarity spreading over interrelated objects.

Web page clustering enhanced by summarization.

Learning similarity measures in non%2Dorthogonal space.
Many machine learning and data mining algorithms crucially rely on the similarity metrics. The Cosine similarity, which calculates the inner product of two normalized feature vectors, is one of the most commonly used similarity measures. However, in many practical tasks such as text categorization and document clustering, the Cosine similarity is calculated under the assumption that the input space is an orthogonal space which usually could not be satisfied due to synonymy and polysemy. Various algorithms such as Latent Semantic Indexing (LSI) were used to solve this problem by projecting the original data into an orthogonal space. However LSI also suffered from the high computational cost and data sparseness. These shortcomings led to increases in computation time and storage requirements for large scale realistic data. In this paper, we propose a novel and effective similarity
Online Supervised Learning for Digital Library.

Improving Text Classification using Local Latent Semantic Indexing.
Latent Semantic Indexing (LSI) has been shown to be extremely useful in information retrieval, but it is not an optimal representation for text classification. It always drops the text classification performance when being applied to the whole training set (global LSI) because this completely unsupervised method ignores class discrimination while only concentrating on representation. Some local LSI methods have been proposed to improve the classification by utilizing class discrimination information. However, their performance improvements over original term vectors are still very limited. In this paper, we propose a new local LSI method called “Local Relevancy Weighted LSI ” to improve text classification by performing a separate Single Value Decomposition (SVD) on the transformed local region of each class. Experimental results show that our method is much better than global LSI and traditional local LSI methods on classification within a much smaller LSI dimension. 1.
IRC An Iterative Reinforcement Categorization Algorithm for Interrelated Web Objects.
Most existing categorization algorithms deal with homogeneous Web data objects, and consider interrelated objects as additional features when taking the interrelationships with other types of objects into account. However, focusing on any single aspects of these interrelationships and objects will not fully reveal their true categories. In this paper, we propose a novel categorization algorithm, the Iterative Reinforcement Categorization algorithm (IRC), to exploit the full interrelationships between the heterogeneous objects on the Web. IRC attempts to classify the interrelated Web objects by iterative reinforcement between individual classification results of different types via the interrelationships. Experiments on a clickthrough log dataset from MSN search engine show that, with the F1 measures, IRC achieves a 26.4 % improvement over a pure content-based classification method, a 21% improvement over a query metadata-based method, and a 16.4 % improvement over a virtual document-based method. Furthermore, our experiments show that IRC converges rapidly. 1.
Mining Ratio Rules Via Principal Sparse Non%2DNegative Matrix Factorization.
Association rules are traditionally designed to capture statistical relationship among itemsets in a given database. To additionally capture the quantitative association knowledge, F.Korn et al recently proposed a paradigm named Ratio Rules [4] for quantifiable data mining. However, their approach is mainly based on Principle Component Analysis (PCA) and as a result, it cannot guarantee that the ratio coefficient is non-negative. This may lead to serious problems in the rules’ application. In this paper, we propose a new method, called Principal Sparse Non-Negative Matrix Factoriza-tion (PSNMF), for learning the associations between itemsets in the form of Ratio Rules. In addition, we provide a support measurement to weigh the importance of each rule for the entire dataset. 1.
Supervised Latent Semantic Indexing for Document Categorization.

Attention model based progressive image transmission.
Progressive image transmission provides a convenient user interface when images are transmitted slowly. However, most of the existing PIT techniques only considered the objective quality of the reconstructed image. Here we present an attention model based progressive image transmission approach to improve the subjective quality of the transmission process. We use both bottom-up image features and top-down semantic information to extract the regions of interest and also propose a new ROI coding scheme based on JPEG2000 to control the trade-off between the transmission of ROI and background. Experiments have shown the efficiency of our approach. 1.
Data%2Ddriven approach for bridging the cognitive gap in image retrieval.

Organizing WWW images based on the analysis of page layout and Web link structure.
Due to the rapid growth of the number of digital images on the Web, there is an increasing demand for effective and efficient method for organizing and retrieving the images available. This paper describes a method for clustering and embedding WWW images. By using a vision-based page segmentation algorithm, a web page is partitioned into blocks, and the textual and link information of an image can be accurately extracted from the block containing that image. By extracting the page-to-block, block-toimage, block-to-page relationships through link structure and page layout analysis, we construct an image graph. With the image graph model, we use techniques from spectral graph theory for image clustering and embedding. Some experimental results are given in the paper. 1.
A content%2Dbased bit allocation model for video streaming.

Extracting texture features from arbitrary%2Dshaped regions for image retrieval.
Lots of work has been done in texture feature extraction for rectangular images, but not as much attention has been paid to the arbitrary-shaped regions available in region-based image retrieval (RBIR) systems. In this paper, we present a texture feature extraction algorithm based on Projection Onto Convex Sets (POCS) theory. POCS iteratively concentrates more and more energy into the selected coefficients from which texture feature of an arbitrary-shaped region can be extracted. Experimental results demonstrate the effectiveness of the proposed algorithm for image retrieval purposes. 1.
Maximizing information throughput for multimedia browsing on small displays.
As a great many of new devices with diverse capabilities are making a population boom, their limited display sizes become the major obstacle that has undermined the usefulness of these devices for information access. In this paper, we introduce our recent research on adapting multimedia content including images, videos and web pages for browsing on small-formfactor devices. A theoretical framework as well as a set of novel methods for presenting and rendering multimedia under limited screen sizes is proposed to improve the user experience. The content modeling and processing are provided as subscriptionbased web services on the Internet. Experiments show that our approach is extensible and able to achieve satisfactory results with high efficiency.
Data%2Ddriven approach for bridging the cognitive gap in image retrieval.

IMMC incremental maximum margin criterion.
Subspace learning approaches have attracted much attention in academia recently. However, the classical batch algorithms no longer satisfy the applications on streaming data or large-scale data. To meet this desirability, Incremental Principal Component Analysis (IPCA) algorithm has been well established, but it is an unsupervised subspace learning approach and is not optimal for general classification tasks, such as face recognition and Web document categorization. In this paper, we propose an incremental supervised subspace learning algorithm, called Incremental Maximum Margin Criterion (IMMC), to infer an adaptive subspace by optimizing the Maximum Margin Criterion. We also present the proof for convergence of the proposed algorithm. Experimental results on both synthetic dataset and real world datasets show that IMMC converges to the similar subspace as that of batch approach.
Combining High Level Symptom Descriptions and Low Level State Information for Configuration Fault Diagnosis.
Automatic fault diagnosis is an important problem for system management. In this paper, we combine high level symptom descriptions and low level state information to solve the system fault diagnosis problem. We extract state-symptom correlation information from knowledge sources in text format, and then use symptom similarity to rank the candidate system states. We apply the method to Windows Registry problems to help Product Support Service (PSS) engineers. Promising results with two different knowledge sources show the robustness of our method. Finally, we explain why this combination is successful and also discuss its limitations.
Adaptive Content Delivery on Mobile Internet across Multiple Form Factor.

Effective browsing of web image search results.
The rapid development of web image search engines has enabled users to search hundred million of images available on the Web. However, due to the unsatisfactory performance of current search technologies, people still need to spend much time in navigating through the large number of result pages to find images of their interest. In this paper, we analyze the user information needs for web image search results browsing and propose to employ a similarity-based organization to present the search results. A user study is carried out to compare our approach with a ranking-based list interface and a cluster-based interface. Experimental results show that similarity-based presentation can help users to explore image search results more naturally and efficiently.
A Novel Gabor%2DLDA Based Face Recognition Method.
Abstract. In this paper, a novel face recognition method based on Gabor-wavelet and linear discriminant analysis (LDA) is proposed. Given training face images, discriminant vectors are computed using LDA. The function of the discriminant vectors is two-fold. First, discriminant vectors are used as a transform matrix, and LDA features are extracted by projecting original intensity images onto discriminant vectors. Second, discriminant vectors are used to select discriminant pixels, the number of which is much less than that of a whole image. Gabor features are extracted only on these discriminant pixels. Then, applying LDA on the Gabor features, one can obtain the Gabor-LDA features. Finally, a combined classifier is formed based on these two types of LDA features. Experimental results show that the proposed method performs better than traditional approaches in terms of both efficiency and accuracy.... 1
Spatio%2Dtemporal Pattern Mining in Sports Video.

Automatic Synchronized Browsing of Images Across Multiple Devices.

Improving Web Browsing on Small Devices Based on Table Classification.

Learning Image Manifold Using Web Data.

Region%2DBased Image Retrieval with Perceptual Colors.
Abstract. Due to the ‘semantic gap ’ between low-level visual features and the rich semantics in user’s mind, performance of traditional contentbased image retrieval systems is far from user’s expectation. In attempt to reduce the ‘semantic gap’, this paper introduces a region-based image retrieval system with high-level semantic color names used. For each segmented region, we define a perceptual color as the low-level color feature of the region. This perceptual color is then converted to a semantic color name. In this way, the system reduces the ‘semantic gap ’ between numerical image features and the richness of human semantics. Four different ways to calculate perceptual color are studied. Experimental results confirm the substantial performance of the proposed system compared to traditional CBIR systems. 1
Salient Region Detection Using Weighted Feature Maps Based on the Human Visual Attention Model.
Abstract. Detection of salient regions in images is useful for object based image retrieval and browsing applications. This task can be done using methods based on the human visual attention model [1], where feature maps corresponding to color, intensity and orientation capture the corresponding salient regions. In this paper, we propose a strategy for combining the salient regions from the individual feature maps based on a new Composite Saliency Indicator (CSI) which measures the contribution of each feature map to saliency. The method also carries out a dynamic weighting of individual feature maps. The experiment results indicate that this combination strategy reflects the salient regions in an image more accurately.

A Linear Approximation Based Method for Noise%2DRobust and Illumination%2DInvariant Image Change Detection.

Learning to cluster web search results.
In web search, surfers are often faced with the problem of selecting their most wanted information from the potential huge amount of search results. The clustering of web search results is the possible solution, but the traditional content based clustering is not sufficient since it ignores many unique features of web pages. The link structure, authority, quality, or trustfulness of search results can play even the higher role than the actual contents of the web pages in clustering. These possible extents are reflected by Google&#039;s PageRank algorithm, HITS algorithm and etc. The main goal of this project is to integrate the authoritative information such as PageRank, link structure (e.g. in-links and out-links) into the K-Means clustering of web search results. The PageRank, inlinks and out-links can be used to extend the vector representation of web pages, and the PageRank can also be considered in the initial centroids selection, or the web page with higher PageRank influences the centroid computation to a higher degree. The relevance of this modified K-Means clustering algorithm needs to be compared to the ones obtained by the content based K-Means clustering, and the effects of different authoritative information also needs to be analyzed.
Web%2Dpage classification through summarization.


Probabilistic model for contextual retrieval.
Contextual retrieval is a critical technique for facilitating many important applications such as mobile search, personalized search, PC troubleshooting, etc. Despite of its importance, there is no comprehensive retrieval model to describe the contextual retrieval process. We observed that incompatible context, noisy context and incomplete query are several important issues commonly existing in contextual retrieval applications. However, these issues have not been previously explored and discussed. In this paper, we propose probabilistic models to address these problems. Our study clearly shows that query log is the key to build effective contextual retrieval models. We also conduct a case study in the PC troubleshooting domain to testify the performance of the proposed models and experimental results show that the models can achieve very good retrieval precision.
Locality preserving indexing for document representation.
Document representation and indexing is a key problem for document analysis and processing, such as clustering, classification and retrieval. Conventionally, Latent Semantic Indexing (LSI) is considered effective in deriving such an indexing. LSI essentially detects the most representative features for document representation rather than the most discriminative features. Therefore, LSI might not be optimal in discriminating documents with different semantics. In this paper, a novel algorithm called Locality Preserving Indexing (LPI) is proposed for document indexing. Each document is represented by a vector with low dimensionality. In contrast to LSI which discovers the global structure of the document space, LPI discovers the local structure and obtains a compact document representation subspace that best detects the essential semantic structure. We compare the proposed LPI approach with LSI on two standard databases. Experimental results show that LPI provides better representation in the sense of semantic structure.

Microsoft Research Asia at Web Track and Terabyte Track of TREC 2004.


Instance%2Dbased Schema Matching for Web Databases by Domain%2Dspecific Query Probing.

Towards Next Generation Web Information Retrieval.

Exploiting PageRank at Different Block Level.
Abstract. In recent years, information retrieval methods focusing on the link analysis have been developed; The PageRank and HITS are two typical ones According to the hierarchical organization of Web pages, we could partition the Web graph into blocks at different level, such as page level, directory level, host level and domain level. On the basis of block, we could analyze the different hyperlinks among pages. Several approaches proposed that the intrahyperlink in a host maybe less useful in computing the PageRank. However, there are no reports on how concretely the intra- or inter-hyperlink affects the PageRank. Furthermore, based on different block level, inter-hyperlink and intra-hyperlink can be two relative concepts. Thus which level should be optimal to distinguish the intra- or inter-hyperlink? And how the ratio set between the intra-hyperlink and inter-hyperlink could ultimately improve performance of the PageRank algorithm? In this paper, we analyze the link distribution at the different block level and evaluate the importance of the intra- and interhyperlink to PageRank on the TREC Web Track data set. Experiment shows that, if we set the block at host level and the ratio of the weight between the intra-hyperlink and inter-hyperlink is 1:4, the retrieval could achieve the best performance. 1
Multi%2Dtype Features Based Web Document Clustering.
Abstract. Clustering has been demonstrated as a feasible way to explore the contents of document collection and organize search engine results. For this task, many features of Web page, such as content, anchor text, URL, hyperlink etc, can be exploited and different results can be obtained. We expect to provide a unified and even better result for end users. Some work have studied how to use several types of features together to perform clustering. Most of them focus on ensemble method or combination of similarity. In this paper, we propose a novel algorithm: Multi-type Features based Reinforcement Clustering (MFRC). This algorithm does not use a unique combine score for all feature spaces, but uses the intermediate clustering result in one feature space as additional information to gradually enhance clustering in other spaces. Finally a consensus can be achieved by such mutual reinforcement. And the experimental results show that MFRC also provides some performance improvement. 1
Optimizing Web Search Using Spreading Activation on the Clickthrough Data.

Learning block importance models for web pages.
Some previous works show that a web page can be partitioned to multiple segments or blocks, and usually the importance of those blocks in a page is not equivalent. Also, it is proved that differentiating noisy or unimportant blocks from pages can facilitate web mining, search and accessibility. But in these works, no uniform approach or model is presented to measure the importance of different portions in web pages. Through a user study, we found that people do have a consistent view about the importance of blocks in web pages. In this paper, we investigate how to find a model to automatically assign importance values to blocks in a web page. We define the block importance estimation as a learning problem. First, we use the VIPS (VIsion-based Page Segmentation) algorithm to partition a web page into semantic blocks with a hierarchy structure. Then spatial features (such as position, size) and content features (such as the number of images and links) are extracted to construct a feature vector for each block. Based on these features, learning algorithms, such as SVM and neural network, are applied to train various block importance models. In our experiments, the best model can achieve the performance with Micro-F1 79 % and Micro-Accuracy 85.9%, which is quite close to a person’s.
Link fusion a unified link analysis framework for multi%2Dtype interrelated data objects.
Web link analysis has proven to be a significant enhancement for quality based web search. Most existing links can be classified into two categories: intra-type links (e.g., web hyperlinks), which represent the relationship of data objects within a homogeneous data type (web pages), and inter-type links (e.g., user browsing log) which represent the relationship of data objects across different data types (users and web pages). Unfortunately, most link analysis research only considers one type of link. In this paper, we propose a unified link analysis framework, called &#034;link fusion&#034;, which considers both the inter- and intra- type link structure among multiple-type inter-related data objects and brings order to objects in each data type at the same time. The PageRank and HITS algorithms are shown to be special cases of our unified link analysis framework. Experiments on an instantiation of the framework that makes use of the user data and web pages extracted from a proxy log show that our proposed algorithm could improve the search effectiveness over the HITS and DirectHit algorithms by 24.6% and 38.2% respectively.
The powerrank web link analysis algorithm.
{byzhang, zhengc,
Affinity rank a new scheme for efficient web search.
Maximizing only the relevance between queries and documents will not satisfy users if they want the top search results to present a wide coverage of topics by a few representative documents. In this paper, we propose two new metrics to evaluate the performance of information retrieval: diversity, which measures the topic coverage of a group of documents, and information richness, which measures the amount of information contained in a document. Then we present a novel ranking scheme, Affinity Rank, which utilizes these two metrics to improve search results. We demonstrate how Affinity Rank works by a toy data set, and verify our method by experiments on real-world data sets.
Similarity spreading a unified framework for similarity calculation of interrelated objects.
In many Web search applications, similarities between objects of one type (say, queries) can be affected by the similarities between their interrelated objects of another type (say, Web pages), and vice versa. We propose a novel framework called similarity spreading to take account of the interrelationship and improve the similarity calculation. Experiment results show that the proposed framework can significantly improve the accuracy of the similarity measurement of the objects in a search engine. Categories &amp; Subject Descriptors:
TSSP A Reinforcement Algorithm to Find Related Papers.
Content analysis and citation analysis are two common methods in recommending system. Compared with content analysis, citation analysis can discover more implicitly related papers. However, the citation-based methods may introduce more noise in citation graph and cause topic drift. Some work combine content with citation to improve similarity measurement. The problem is that the two features are not used to reinforce each other to get better result. To solve the problem, we propose a new algorithm, Topic Sensitive Similarity Propagation (TSSP), to effectively integrate content similarity into similarity propagation. TSSP has two parts: citation context based propagation and iterative reinforcement. First, citation contexts provide clues for which papers are topic related to and filter out less irrelevant citations. Second, iteratively integrating content and citation similarity enable them to reinforce each other during the propagation. The experimental results of a user study show TSSP outperforms other algorithms in almost all cases. 1.
GE%2DCKO A Method to Optimize Composite Kernels for Web Page Classification.
Most of current researches on Web page classification focus on leveraging heterogeneous features such as plain text, hyperlinks and anchor texts in an effective and efficient way. Composite kernel method is one topic of interest among them. It first selects a bunch of initial kernels, each of which is determined separately by a certain type of features. Then a classifier is trained based on a linear combination of these kernels. In this paper, we propose an effective way to optimize the linear combination of kernels. We proved that this problem is equivalent to solving a generalized eigenvalue problem. And the weight vector of the kernels is the eigenvector associated with the largest eigenvalue. A support vector machine (SVM) classifier is then trained based on this optimized combination of kernels. Our experiment on the WebKB dataset has shown the effectiveness of the proposed method. 1.
Learning important models for web page blocks based on layout and content analysis.
Previous work shows that a web page can be partitioned into multiple segments or blocks, and often the importance of those blocks in a page is not equivalent. It has also been proven that differentiating noisy and unimportant blocks from pages can facilitate web mining, search and accessibility. However, no uniform approach and model has been presented to measure the importance of different blocks in a web page. Through a user study, we found that people do have a consistent view about the importance of blocks in a web page. Thus, we investigate how to find a model to automatically assign importance values to blocks in a web page. We formulate the block importance estimation as a learning problem. First, we use a vision-based page segmentation technique to partition a web page into semantic blocks with a hierarchical structure. Then spatial features (such as position and size) and content features (such as the number of images and links) are extracted to construct a feature vector for each block. Then, learning algorithms are used to train a model to assign importance to each block in the web page. In our experiments, the best model can achieve the performance with Micro-F1 80.2 % and Micro-
MobiPicture browsing pictures on mobile devices.

Automatic browsing of large pictures on mobile devices.
Pictures have become increasingly common and popular in mobile communications. However, due to the limitation of mobile devices, there is a need to develop new technologies to facilitate the browsing of large pictures on the small screen. In this paper, we propose a novel approach which is able to automate the scrolling and navigation of a large picture with a minimal amount of user interaction on mobile devices. An image attention model is employed to illustrate the information structure within an image. An optimal image browsing path is then calculated based on the image attention model to simulate the human browsing behaviors. Experimental evaluations of the proposed mechanism indicate that our approach is an effective way for viewing large images on small displays.
Looking into video frames on small displays.
With the growing popularity of personal digital assistants and smart phones, people have become enthusiastic to watch videos through these mobile devices. However, a crucial challenge is to provide a better user experience for browsing videos on the limited and heterogeneous screen sizes. In this paper, we present a novel approach which allows users to overcome the display constraints by zooming into video frames while browsing. An automatic approach for detecting the focus regions is introduced to minimize the amount of user interaction. In order to improve the quality of output stream, virtual camera control is employed in the system. Preliminary evaluation shows that this approach is an effective way for video browsing on small displays.
Video summarization based on user log enhanced link analysis.
Efficient video data management calls for intelligent video summarization tools that automatically generate concise video summaries for fast skimming and browsing. Traditional video summarization techniques are based on low-level feature analysis, which generally fails to capture the semantics of video content. Our vision is that users unintentionally embed their understanding of the video content in their interaction with computers. This valuable knowledge, which is difficult for computers to learn autonomously, can be utilized for video summarization process. In this paper, we present an intelligent video browsing and summarization system that utilizes previous viewers&#039; browsing log to facilitate future viewers. Specifically, a novel ShotRank notion is proposed as a measure of the subjective interestingness and importance of each video shot. A ShotRank computation framework is constructed to seamlessly unify low-level video analysis and user browsing log mining. The resulting ShotRank is used to organize the presentation of video shots and generate video skims. Experimental results from user studies have strongly confirmed that ShotRank indeed represents the subjective notion of interestingness and importance of each video shot, and it significantly improves future viewers&#039; browsing experience.
Knowing a tree from the forest art image retrieval using a society of profiles.
This paper aims to address the problem of art image retrieval (AIR), which aims to help users find their favorite painting images. AIR is of great interests to us because of its application potentials and interesting research challenges--- the retrieval is not only based on painting contents or styles, but also heavily based on user preference profiles. This paper describes the collaborative ensemble learning, a novel statistical learning approach to this task. It at first applies probabilistic support vector machines (SVMs) to model each individual user&#039;s profile based on given examples, i.e. liked or disliked paintings. Due to the high complexity of profile modelling, the SVMs can be rather weak in predicting preferences for new paintings. To overcome this problem, we combine a society of users&#039; profiles, represented by their respective SVM models, to predict a given user&#039;s preferences for painting images. We demonstrate that the combination scheme is embedded in a Bayesian framework and retains intuitive interpretations---like-minded users are likely to share similar preferences. We report extensive empirical studies based on two experimental settings. The first one includes some controlled simulations performed on 4533 painting images. In the second setting, we report evaluations based on user preferences collected through an online web-based survey. Both experiments demonstrate that the proposed approach achieves excellent performance in terms of capturing a user&#039;s diverse preferences.
Extracting Content Structure for Web Pages Based on Visual Representation.
Abstract. A new web content structure based on visual representation is proposed in this paper. Many web applications such as information retrieval, information extraction and automatic page adaptation can benefit from this structure. This paper presents an automatic top-down, tag-tree independent approach to detect web content structure. It simulates how a user understands web layout structure based on his visual perception. Comparing to other existing techniques, our approach is independent to underlying documentation representation such as HTML and works well even when the HTML structure is far different from layout structure. Experiments show satisfactory results. 1
CBC Clustering Based Text Classification Requiring Minimal Labeled Data.
Semi-supervised learning methods construct classifiers using both labeled and  unlabeled training data samples. While unlabeled data samples can help to improve the accuracy  of trained models to certain extent, existing methods still face difficulties when labeled data is  not sufficient and biased against the underlying data distribution. In this paper, we present a  clustering based classification (CBC) approach. Using this approach, training data, including  both the labeled and unlabeled data, is first clustered with the guidance of the labeled data. Some  of unlabeled data samples are then labeled based on the clusters obtained. Discriminative  classifiers can subsequently be trained with the expanded labeled dataset. The effectiveness of  the proposed method is justified analytically. Related issues such as expanding labeled dataset  and interacting clustering with classification are discussed. Our experimental results  demonstrated that CBC outperforms existing algorithms when the size of labeled dataset is very  small.
An Evaluation on Feature Selection for Text Clustering.
Feature selection methods have been successfully applied to text categorization but seldom applied to text clustering due to the unavailability of class label information. In this paper, we first give empirical evidence that feature selection methods can improve the efficiency and performance of text clustering algorithm. Then we propose a new feature selection method called “Term Contribution (TC) ” and perform a comparative study on a variety of feature selection methods for text clustering, including Document Frequency (DF), Term Strength (TS), Entropy-based (En), Information Gain (IG) and א 2 statistic (CHI). Finally, we propose an “Iterative Feature Selection (IF) ” method that addresses the unavailability of label problem by utilizing effective supervised feature selection method to iteratively select features and perform clustering. Detailed experimental results on Web Directory data are provided in the paper. 1.
Image Adaptation Based on Attention Model for Small%2DForm%2DFactor Device.
Image adaptation, one of the essential problems in adaptive content delivery for universal access, has been actively explored for some time. Most existing approaches have focused on generic adaptation towards saving file size under constraints in client environment and hardly paid attention to user’s perception on the adapted result. Meanwhile, the major limitation on the user’s delivery context is moving from data volume (or time-to-wait) to screen size because of the galloping development of hardware technologies. In this paper, we propose a novel method for adapting images based on user attention. A generic and extensible image attention model is introduced based on three attributes (region of interest, attention value, and minimal perceptible size) associated with each attention object. An algorithm is developed to find the optimal adaptation efficiently. Experimental results demonstrate the usefulness of the proposed scheme and its potential application in the future. 1
ReCoM reinforcement clustering of multi%2Dtype interrelated data objects.
Most existing clustering algorithms cluster highly related data objects such as Web pages and Web users separately. The interrelation among different types of data objects is either not considered, or represented by a static feature space and treated in the same ways as other attributes of the objects. In this paper, we propose a novel clustering approach for clustering multi-type interrelated data objects, ReCoM (Reinforcement Clustering of Multi-type Interrelated data objects). Under this approach, relationships among data objects are used to improve the cluster quality of interrelated data objects through an iterative reinforcement clustering process. At the same time, the link structure derived from relationships of the interrelated data objects is used to differentiate the importance of objects and the learned importance is also used in the clustering process to further improve the clustering results. Experimental results show that the proposed approach not only effectively overcomes the problem of data sparseness caused by the high dimensional relationship space but also significantly improves the clustering accuracy.
Building a web thesaurus from web link structure.
Thesaurus has been widely used in many applications, including information retrieval, natural language processing, and question answering. In this paper, we propose a novel approach to automatically constructing a domain-specific thesaurus from the Web using link structure information. The proposed approach is able to identify new terms and reflect the latest relationship between terms as the Web evolves. First, a set of high quality and representative websites of a specific domain is selected. After filtering out navigational links, link analysis is applied to each website to obtain its content structure. Finally, the thesaurus is constructed by merging the content structures of the selected websites. The experimental results on automatic query expansion based on our constructed thesaurus show 20% improvement in search precision compared to the baseline.
Implicit link analysis for small web search.

Microsoft Research Asia at the Web Track of TREC 2003.

Collaborative Ensemble Learning Combining Collaborative and Content%2DBased Information Filtering via Hierarchical Bayes.
Collaborative filtering (CF) and contentbased filtering (CBF) have widely been used in information filtering applications, both approaches having their individual strengths and weaknesses. This paper proposes a novel probabilistic framework to unify CF and CBF, named collaborative ensemble learning. Based on content based probabilistic models for each user’s preferences (the CBF idea), it combines a society of users ’ preferences to predict an active user’s preferences (the CF idea). While retaining an intuitive explanation, the combination scheme can be interpreted as a hierarchical Bayesian approach in which a common prior distribution is learned from related experiments. It does not require a global training stage and thus can incrementally incorporate new data. We report results based on two data sets, the Reuters-21578 text data set and a data base of user opionions on art images. For both data sets, collaborative ensemble achieved excellent performance in terms of recommendation accuracy. In addition to recommendation engines, collaborative ensemble learning is applicable to problems typically solved via classical hierarchical Bayes, like multisensor fusion and multitask learning. 1
Improving pseudo%2Drelevance feedback in web information retrieval using web page segmentation.
In contrast to traditional document retrieval, a web page as a whole is not a good information unit to search because it often contains multiple topics and a lot of irrelevant information from navigation, decoration, and interaction part of the page. In this paper, we propose a VIsion-based Page Segmentation (VIPS) algorithm to detect the semantic content structure in a web page. Compared with simple DOM based segmentation method, our page segmentation scheme utilizes useful visual cues to obtain a better partition of a page at the semantic level. By using our VIPS algorithm to assist the selection of query expansion terms in pseudo-relevance feedback in web information retrieval, we achieve 27 % performance improvement on Web Track dataset.
Detecting web page structure for adaptive viewing on small form factor devices.
Mobile devices have already been widely used to access the Web. However, because most available web pages are designed for desktop PC in mind, it is inconvenient to browse these large web pages on a mobile device with a small screen. In this paper, we propose a new browsing convention to facilitate navigation and reading on a small-form-factor device. A web page is organized into a two level hierarchy with a thumbnail representation at the top level for providing a global view and index to a set of subpages at the bottom level for detail information. A page adaptation technique is also developed to analyze the structure of an existing web page and split it into small and logically related units that fit into the screen of a mobile device. For a web page not suitable for splitting, auto-positioning or scrolling-by-block is used to assist the browsing as an alterative. Our experimental results show that our proposed browsing convention and developed page adaptation scheme greatly improve the user’s browsing experiences on a device with a small display.
DRESS A Slicing Tree Based Web Page Representation for Various Display Sizes.

Media Companion Delivering Content%2Doriented Web Services to Internet Media.

User Access Pattern Enhanced Small Web Search.

Learning a semantic space from user%27s relevance feedback for image retrieval.

Query Expansion by Mining User Logs.
Queries to search engines on the Web are usually short. They do not provide sufficient evidence for an effective selection  of relevant documents. Previous research has proposed the utilization of query expansion to deal with this problem. However,  expansion terms are usually determined on term co-occurrences within documents. In this study, we propose a new method for query  expansion based on user interactions recorded in user logs. The central idea is to extract correlations between query terms and  document terms by analyzing user logs. These correlations are then used to select high-quality expansion terms for new queries. Compared to
Ubiquitous media agents a framework for managing personally accumulated multimedia files.

A visual attention model for adapting images on small displays.

Alternating Feature Spaces in Relevance Feedback.
Image retrieval using relevance feedback can be considered as a classification process. In practice, the generalization of classifier is often constrained by the insufficiency of training samples. In this paper, we propose a novel relevance feedback approach capable of collecting more representative samples. Image labeling and classifier training are conducted in two complementary image feature spaces. The complementarities between feature spaces are also studied. Our experimental result based on 10,000 images indicates that the proposed approach significantly improves image retrieval performance.
A Multi%2Dparadigm Querying Approach for a Generic Multimedia Database Management System.
To truly meet the requirements of multimedia  database (MMDB) management, an integrated  framework for modeling, managing and retrieving  various kinds of media data in a uniform way is  necessary. MediaLand is an experimental MMDB  platform being developed at Microsoft Research  Asia for users with different levels of experiences  and expertise to manage and search multimedia  repositories easily, efficiently, and cooperatively.
Enabling personalization services on the edge.

Learning and inferring a semantic space from user%27s relevance feedback for image retrieval.

Visual Based Content Understanding towards Web Adaptation.
Web content structure is proposed to facilitate automatic web page  adaptation in this paper. By identifying the logic relationship of web content  based on layout information, web content structure effectively represents authors  &#039; presentation intention. An automatic top-down, tag-tree independent approach  to detect web content structure is presented. It simulates how a user understands  web layout structure based on his vision. Comparing to other content  analysis techniques, our approach is independent to physical realization and  works well even when the physical structure is far different from layout structure.
Improving Image Retrieval with Semantic Classification Using Relevance Feedback.

Ranking user%27s relevance to a topic through link analysis on web logs.
Computing the web-user&#039;s relevance to a give topic is an important task for any personalization service on the Web. Since the interest and preference of a web-user are revealed in his Web browsing history, in this paper we develop a novel approach that utilizes Web logs to compute the relevance of a web-user to a given query. In contrast to traditional methods that are purely based on textual analysis, our approach calculates the web-user&#039;s relevance through link analysis under a unified framework where the importance of web-pages and web-users mutually reinforce each other in an iterative way. The experimental results show that our approach has achieved 53% of accuracy when ranking the web-user&#039;s relevance to a search topic.
A Unified Framework for Clustering Heterogeneous Web Objects.
In this paper, we introduce a novel framework for clustering web data which is often heterogeneous in nature. As most existing methods often integrate heterogeneous data into a unified feature space, their flexibilities to explore and adjust contributing effect from different heterogeneous information are compromised. In contrast, our framework enables separate clustering of homogeneous data in the entire process based on their respective features, and a layered structure with link information is used to iteratively project and propagate the clustered results between layers until it converges. Our experimental results show that such a scheme not only effectively overcomes the problem of data sparseness caused by the high dimensional link space but also improves the clustering accuracy significantly. We achieve 19% and 41% performance increases when clustering web-pages and users based on a semi-synthetic web log. Finally, we show a real clustering result based on UC Berkeley&#039;s web log.
A Unified Framework for Web Link Analysis.
Web link analysis has been proved to provide significant enhancement to the precision of web search in practice. Among existing approaches, Kleinberg&#039;s HITS and Google&#039;s PageRank are the two most representative algorithms that employ explicit hyperlinks structure among web pages to conduct link analysis, and DirectHit represents the other extreme that takes the user&#039;s access frequency as implicit link to the web page for counting its importance. In this paper, we propose a novel link analysis algorithm which puts both explicit and implicit link structures under a unified framework, and show that HITS and DirectHit are essentially the two extreme instances of our proposed method. One important advantage of our method is its ability to analyze not only the hyperlinks between web-pages but also the interactions between the users and the Web at the same time. The importance of web-pages and users can reinforce each other to improve the Web link analysis. Compared with traditional HITS and DirectHit algorithms, our method further improves the search precision by 11.8% and 25.3%.
Log Mining to Improve the Performance of Site Search.
Despite of the popularity of global search engines, people still suffer from low accuracy of site search. The primary reason lies in the difference of link structures and data scale between global Web and website, which leads&#039; to failures of traditional re-ranking methods&#039; such as HITS, PageRank and DirectHit. This&#039; paper proposes a novel re-ranking method based on user logs within websites. With the help of website taxonomy, we mine for generalized association rules and abstract access patterns of different levels&#039;. Mining results&#039; are subsequently used to re-rank the retrieved pages. One of  the advantages of our mining algorithm is that it resolves the diversity problem of user&#039;s&#039; access behavior and discovers&#039; general patterns. Experiment shows that the proposed method outperforms keyword-based method by 15% and DirectHit by 13% respectively.
Probabilistic query expansion using query logs.
Query expansion has long been suggested as an effective way to resolve the short query and word mismatching problems. A number of query expansion methods have been proposed in traditional information retrieval. However, these previous methods do not take into account the specific characteristics of web searching; in particular, of the availability of large amount of user interaction information recorded in the web query logs. In this study, we propose a new method for query expansion based on query logs. The central idea is to extract probabilistic correlations between query terms and document terms by analyzing query logs. These correlations are then used to select high-quality expansion terms for new queries. The experimental results show that our log-based probabilistic query expansion method can greatly improve the search performance and has several advantages over other existing methods.
An interactive video delivery and caching system using video summarization.
With the advance of high-speed network technologies, the availability and popularity of streaming media content over the Internet has grown rapidly in recent years. The delivery and caching of streaming media must be handled in a different fashion than that of traditional non-streaming objects such as HTML or image files, because of its distinct characteristics and user viewing patterns. We propose a novel scheme that provides users with the video summary (a number of key-frame images) before they download the file, and options for them to select the starting playback position. We introduce the content analysis service to achieve these functionalities. The video content analysis performs shot boundary detection, key-frame selection, and face detection and tracking. The results of the processing are a segmented video sequence and an XML-based meta-data describing the video content. We also design a caching system that utilizes our video abstraction and summarization technique. Our integrated video delivery and caching system combines content-aware segmentation, prefix caching, prefetching, and cooperative caching. We describe how our scheme can be applied in three proposed caching architectures.
User Intention Modelling in Web Applications Using Data Mining.
The problem of inferring a user&#039;s intentions in Machine--Human Interaction has been the key research issue for providing personalized experiences and services. In this paper, we propose novel approaches on modeling and inferring user&#039;s actions in a computer. Two linguistic features -- keyword and concept features -- are extracted from the semantic context for intention modeling. Concept features are the conceptual generalization of keywords. Association rule mining is used to find the proper concept of corresponding keyword. A modified Nave Bayes classifier is used in our intention modeling. Experimental results have shown that our proposed approach achieved 84% average accuracy in predicting user&#039;s intention, which is close to the precision (92%) of human prediction.
Learning Similarity Measure for Natural Image Retrieval with Relevance Feedback.
A new scheme of learning similarity measure is proposed for content-based image retrieval (CBIR). It learns a boundary that separates the images in the database into two clusters. Images inside the boundary are ranked by their Euclidean distances to the query. The scheme is called constrained similarity measure (CSM), which not only takes into consideration the perceptual similarity between images, but also significantly improves the retrieval performance of the Euclidean distance measure. Two techniques, support vector machine (SVM) and AdaBoost from machine learning, are utilized to learn the boundary. They are compared to see their differences in boundary learning. The positive and negative examples used to learn the boundary are provided by the user with relevance feedback. The CSM metric is evaluated in a large database of 10 009 natural images with an accurate ground truth. Experimental results demonstrate the usefulness and effectiveness of the proposed similarity measure for image retrieval.
EdgeFlow a technique for boundary detection and image segmentation.
Abstract—A novel boundary detection scheme based on “edge flow ” is proposed in this paper. This scheme utilizes a predictive coding model to identify the direction of change in color and texture at each image location at a given scale, and constructs an edge flow vector. By propagating the edge flow vectors, the boundaries can be detected at image locations which encounter two opposite directions of flow in the stable state. A user defined image scale is the only significant control parameter that is needed by the algorithm. The scheme facilitates integration of color and texture into a single framework for boundary detection. Segmentation results on a large and diverse collections of natural images are provided, demonstrating the usefulness of this method to content based image retrieval. Index Terms—Boundary detection, Gabor filtering, image segmentation, texture.
Blur Determination in the Compressed Domain Using DTC Information.

NeTra A Toolbox for Navigating Large Image Databases.
. We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as &#034;retrieve all images that contain regions that have the colo...
A Texture Thesaurus for Browsing Large Aerial Photographs.
this article, we will demonstrate that texture could large-scale aerial photographs is presented. The salient be used to select a large number of geographically salient components of this system include texture feature ex- features including vegetation patterns, parking lots, and traction, image segmentation and grouping, learning building developments. Using texture primitives as visual similarity measure, and a texture thesaurus model for features, one can query the database to retrieve similar fast search and indexing. The texture features are computed by filtering the image with a bank of Gabor filters. image patterns. Much of the results presented are with This is followed by a texture gradient computation to airphotos, although a similar analysis can be applied to segment each large airphoto into homogeneous regions. LANDSAT and SPOT satellite images. A schematic dia- A
Edge Flow A Framework of Boundary Detection and Image Segmentation.
A novel boundary detection scheme based on &#034;edge flow&#034; is proposed in this paper. This scheme utilizes a predictive coding model to identify the direction of change in color and texture at each image location at a given scale, and constructs an edge flow vector. By iteratively propagating the edge flow, the boundaries can be detected at image locations which encounter two opposite directions of flow in the stable state. A user defined image scale is the only significant control parameter that is needed by the algorithm. The scheme facilitates integration of color and texture into a single framework for boundary detection.  1 Introduction  In most computer vision applications, the edge/boundary detection and image segmentation constitute a crucial initial step before performing high-level tasks such as object recognition and scene interpretation. However, despite considerable research and progress made in this area, the robustness and generality of the algorithms on large image datasets...
NeTra A Toolbox for Navigating Large Image Databases.
. We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as &#034;retrieve all images that contain regions that have the colo...
Texture Features and Learning Similarity.

Texture Features for Browsing and Retrieval of Image Data.

Texture%2DBased Pattern Retrieval from Image Databases.

A comparison of wavelet transform features for texture image annotation.

Adaptive Filtering and Indexing for Image Databases.
In this paper we combine image feature extraction with indexing techniques for efficient retrieval in large texture images databases. A 2D image signal is processed using a set of Gabor filters to derive a 120 component feature vector representing the image. The feature components are ordered based on the relative importance in characterizing a given texture pattern, and this facilitates the development of efficient indexing mechanisms. We propose three different sets of indexing features based on the &#034;best feature&#034;, the average feature and a combination of both. We investigate the tradeoff between accuracy and discriminating power using these different indexing approaches, and conclude that the combination of &#034;best feature&#034; and the average feature gives the best results.  Keywords: Texture Images, Gabor Filters, Multi-dimensional Indexing, Indexing Structures. 1 INTRODUCTION  Searching a large database of image data poses many challenging problems. In particular, issues related to ima...
